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Abstract 


Compositional  reasoning  is  an  approach  for  scaling  model  checking  to  complex  com¬ 
puter  systems,  where  a  given  property  of  a  system  is  decomposed  into  properties 
of  small  parts  of  the  system.  The  key  difficulty  with  compositional  reasoning  is  in 
automatically  coming  up  with  sufficient  decompositions  of  global  properties  into  lo¬ 
cal  properties.  This  thesis  develops  efficient  compositional  algorithms  for  safety  of 
(a)  sequential  recursive  programs,  using  solvers  for  SAT  and  SAT  modulo  theories 
(SMT),  and  (b)  parallel,  finite-state  probabilistic  systems.  These  algorithms  result 
in  significant  improvements  over  the  state-of-the-art,  both  in  theory  and  in  practice. 

For  SAT-based  verification  of  sequential  programs,  monolithic  techniques  based 
on  Bounded  Model  Checking  (BMC)  iteratively  check  satisfiability  of  formulas  whose 
size  can  grow  exponentially  in  the  input  size  of  the  program.  While  safety  can  be 
decided  in  time  polynomial  in  the  number  of  states,  existing  SAT-based  algorithms 
do  not  have  such  guarantees.  We  develop  a  compositional  SAT-based  algorithm 
that  maintains  and  utilizes  under-  and  over- approximations  of  the  behavior  of  pro¬ 
cedures.  While  addressing  the  above  complexity  problem,  the  algorithm  also  extends 
to  realistic  programs  that  involve  arithmetic  operations  using  oracles  for  SMT. 

In  order  to  improve  practical  convegence  of  the  iterative  approach  for  SMT-based 
verification,  we  also  develop  a  new  mechanism  for  automatic  abstraction  refinement 
of  the  input  program.  This  combines  ideas  from  Proof  Based  Abstraction  (PBA)  and 
Counter  Example  Guided  Abstraction  Refinement  (CEGAR)  in  the  literature. 

We  describe  Spacer  ( Software  Proof-based  Abstraction  with  CounterExample- 
based  Refinement ),  a  tool  that  implements  the  above  algorithms,  using  which  we 
show  significant  advantages  on  realistic  benchmarks. 

For  probabilistic  transition  systems  with  multiple  parallel  components,  the  num¬ 
ber  of  states  of  a  system  can  grow  exponentially  in  the  number  of  components  (the 
well-known  state-space  explosion  problem).  For  these  systems,  we  develop  the  Erst 
compositional  algorithms  for  checking  simulation  conformance.  We  follow  an  assume- 
guarantee  style  reasoning  and  establish  theoretical  bounds  on  the  learnability  of  an 
intermediate  assumption  of  the  least  number  of  states  from  positive  and  negative 
examples.  We  also  develop  a  practical  algorithm  based  on  abstraction  refinement. 
Using  a  Java  implementation  of  the  latter,  we  show  practical  advantage  over  mono¬ 
lithic  verification. 
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Chapter  1 


Introduction 


Model  checking  [39]  is  an  automatic  technique  for  verifying  correctness  properties 
of  computer  systems.  Since  its  invention  in  the  1980’s,  numerous  approaches  have 
been  proposed  for  scaling  model  checking  to  complex  and  real-world  systems.  We 
are  interested  in  the  approach  of  compositional  reasoning,  where  the  basic  idea  is  to 
decompose  a  given  property  of  a  system  into  properties  of  small  parts  of  the  system. 
If  the  local  properties  together  imply  the  overall  property  of  the  system,  it  suffices  to 
check  each  of  the  local  properties.  Such  an  approach  can  be  very  efficient  in  practice 
as  the  whole  system  can  be  exponentially  more  complex  than  the  individual  parts 
combined.  For  example,  the  size  of  the  state-space  of  a  reactive  system  composed 
of  multiple  components  running  in  parallel  can  grow  exponentially  in  the  number  of 
components  and  this  phenomenon  is  well-known  as  the  state-space  explosion.  Sev¬ 
eral  frameworks  have  been  developed  for  compositional  reasoning  of  such  reactive 
systems  (e.g.,  [35,  98]).  In  the  context  of  program  verification,  Hoare  logic  [73],  in 
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particular  the  Rule  of  Composition  and  the  Rule  of  Recursion ,  can  also  be  seen  as 
a  compositional  framework  for  checking  partial  correctness  triples.  Here,  a  local 
property  corresponds  to  a  Hoare  triple  for  an  individual  statement  or  a  procedure. 
With  Hoare  logic,  it  suffices  to  find  one  generic  local  property  per  procedure  that 
can  be  adapted  to  analyze  every  call  to  the  procedure  [37],  despite  the  possibility  of 
exponentially  many  such  calls  in  an  execution  of  the  program  it  is  part  of.  However, 
the  main  challenge  in  compositional  reasoning  is  to  automatically  come  up  with  a 
sufficient  decomposition  of  an  overall  property  of  the  system  into  local  properties.  In 
this  thesis,  we  develop  several  efficient  algorithms  for  automatically  discovering  such 
decompositions  to  check  safety  of  recursive  programs  and  probabilistic  systems  with 
multiple,  parallel  components. 


1.1  SMT-Based  Software  Model  Checking 

The  first  step  in  software  model  checking  is  to  identify  the  logical  systems  used  to 
model  the  various  program  operations  and  express  predicates  describing  the  pro¬ 
gram’s  behavior.  In  this  thesis,  we  use  first-order  languages  for  these  purposes.  In 
particular,  we  are  interested  in  model  checking  techniques  that  are  based  on  checking 
satisfiability  of  logical  formulas  in  the  languages. 

The  introduction  of  Boolean  satisfiability  (SAT)  solvers  in  model  checking  has 
revolutionized  the  held  and  SAT-based  algorithms  are  some  of  the  best  we  have 
today.  The  idea  of  using  a  SAT  solver  for  model  checking  was  first  introduced  by 
Biere  et  al.  [22]  using  a  technique  called  Bounded  Model  Checking  (BMC).  BMC  was 
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proposed  for  the  verification  of  a  (symbolically  represented)  Kripke  structure  against 
temporal  logic  specifications  for  a  given  bound  on  the  length  of  a  counterexample. 

In  order  to  extend  SAT-based  methods  to  software  model  checking,  one  needs  an 
oracle  for  satisfiability  of  formulas  in  the  underlying  first-order  language.  However,  as 
allowing  arbitrary  interpretations  of  the  various  program  operations  is  undesirable, 
one  typically  utilizes  a  first-order  theory  of  sentences  to  characterize  the  intended 
interpretation  of  the  operations.  For  example,  Presburger  Arithmetic  characterizes 
linear  arithmetic  over  integers.  Thus,  we  need  an  oracle  for  satisfiability  modulo  theo¬ 
ries  (SMT).  Then,  to  obtain  a  BMC  procedure  for  safety  in  sequential  programs,  the 
bound  is  typically  on  the  number  of  loop  iterations  (e.g.,  see  [  10])  and  on  the  depth  of 
recursion1  (e.g.,  see  [7,  92]),  which  implicitly  bounds  the  length  of  a  counterexample. 
In  other  words,  the  bound  h  corresponds  to  all  executions  that  make  at  most  h  nested 
calls  and  that  use  at  most  b  iterations  of  any  given  loop.  We  use  the  term  bounded 
safety  to  refer  to  the  problem  of  checking  safety  for  a  given  bound  (in  the  above 
sense;  see  Chapters  2,  3  for  details).  To  prove  safety,  we  use  Hoare  triples  for  partial 
correctness  specifications.  Here,  given  assertions  ip,  if  in  the  underlying  first-order 
language  and  a  statement  r,  a  Hoare  triple  {<p}  r  {if}  specifies  that  whenever  r  is 
executed  from  a  state  satisfying  Lp ,  it  will  either  fail  to  terminate  or  end  in  a  state 
satisfying  if.  There  exist  sound  and  complete  (in  the  sense  of  Cook  [  11])  Hoare  proof 
systems  for  while  programs  [73]  and  procedural  programs  [36,  37]. 


1  In  general,  a  procedure  can  call  other  procedures,  perhaps  in  a  mutually  recursive  way,  in  which 
case  bounding  the  call-stack  leads  to  a  more  systematic  approach.  See  Chapter  2  for  details. 
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Main  ()  { 

bool  b  :=  nd() ; 
Level<l>  (b) ; 
Level<l>  (b) ; 
assert  (b) ; 

} 


Level<i>  (bool  b)  { 
if  (!b)  { 

Level<i+1>  (b) ; 
Level<i+1>  (b) ; 

> 

b  :=  !b; 

} 


Main 

Level<l>  Level<l> 


Level<ri,> 


Level<ro> 


Figure  1.1:  A  Boolean  program  with  exponential  unwinding  size. 


1.1.1  Efficient  Bounded  Safety 


Existing  BMC  algorithms  for  bounded  safety  create  SMT  problems  that  may  grow 
exponentially  with  the  bound  on  the  recursion  depth  due  to  the  tree-like  unrolling 
of  the  call-graph.  For  example,  Fig.  1.1  shows  a  program  with  Boolean  variables 
(adapted  from  [19])  and  finitely  many  Level<i>  procedures.  Here,  nd  is  a  routine 
that  returns  an  unknown  Boolean  value.2  For  a  bound  n  on  the  number  of  such 
procedures,  assuming  that  procedure  calls  in  the  body  of  Level<n>  are  replaced  by 
noops,  the  figure  also  shows  its  tree-like  unrolling  which  grows  exponentially  in  n. 
With  one  Boolean  parameter  per  procedure,  note  that  the  number  of  program  states 
is  linear  in  n,  where  a  state  corresponds  to  a  valuation  of  the  program  counter  and 
the  variables  in  scope.  Therefore,  many  of  present-day  BMC-based  model  checking 
algorithms,  e.g.,  Whale  [  ],  HSF  [63],  Ultimate  Automizer  [68,  69],  Duality  [92], 
are  at  least  worst-case  exponential  in  the  number  of  states  for  Boolean  programs. 
However,  note  that  the  operational  semantics  of  a  Boolean  program  (with  the  crucial 
assumption  that  procedures  are  disallowed  as  parameters  [36])  can  be  defined  in  terms 
of  a  pushdown  automaton  where  the  push  and  pop  operations  on  the  stack  correspond 

2In  other  words,  assume  that  the  behavior  of  nd  is  unknown.  So,  for  the  purpose  of  verification, 
nd  effectively  returns  either  true  or  false  non-deterministically. 
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to  procedure  calls  and  returns,  and  the  accepting  states  denote  the  safe  program 
states.  This  reduces  safety  in  Boolean  programs  to  state-reachability  in  pushdown 
automata  and  there  exist  polynomial-time  (cubic)  algorithms  for  the  latter  that  are 
not  SAT-based  [11,  19,  101]. 

On  the  other  hand,  the  algorithm  GPDR  [  ]  follows  the  approach  of  IC3  [25] 

by  solving  BMC  incrementally  without  unrolling  the  call-graph.  For  some  configu¬ 
rations  (e.g.,  explicit-state  reasoning),  GPDR  is  worst-case  polynomial  for  Boolean 
Programs.  However,  it  gets  more  challenging  when  the  program  operations  and  for¬ 
mulas  are  in  a  first-order  language.  In  this  case,  GPDR  might  even  fail  to  find  a 
counterexample  despite  the  presence  of  an  SMT  oracle,  unlike  the  guarantee  given  by 
other  BMC-based  algorithms  mentioned  above  (see  Appendix  2. A  for  an  example). 

To  address  the  aforementioned  problems,  we  present  a  new  SMT-based  algorithm 
for  analyzing  the  program  compositionally.  That  is,  we  iteratively  check  safety  prop¬ 
erties  of  individual  procedures  and  infer  approximations  about  their  input-output 
behavior,  by  making  use  of  the  previously  inferred  approximations  of  the  procedures 
being  called.  Our  main  insight  is  to  utilize  not  only  over-approximations  of  procedure 
behaviors,  as  in  existing  algorithms,  but  also  their  under- approximations. 

For  Boolean  Programs,  this  results  in  a  terminating  algorithm  for  safety,  without 
any  bound,  and  has  a  polynomial  time  complexity  (see  Chapter  2).  Moreover,  in 
general,  assuming  an  SMT  oracle  for  the  first-order  language  of  the  assertions  and 
the  program  operations,  we  show  that  our  compositional  algorithm  terminates  for 
bounded  safety.  To  the  best  of  our  knowledge,  this  is  the  first  SMT-based  algorithm 
which  such  guarantees.  Details  are  discussed  in  Chapter  2. 
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1.1.2  Better  Proofs  of  Bounded  Safety 


While  bounded  safety  of  programs  is  reducible  to  SMT  (via  BMC),  unbounded  safety 
of  programs  is  undecidable  in  general.3  Many  of  present-day  algorithms  for  SMT- 
based  model  checking  follow  an  iterative  approach  by  checking  bounded  safety  for 
increasing  values  of  the  bound  on  the  loop  iterations  and  the  recursion  depth.  Each 
iteration  of  these  algorithms  corresponds  to  checking  whether  there  is  a  counterex¬ 
ample  to  safety  for  the  given  bound  on  the  executions  by  means  of  an  SMT  solver 
(using  BMC).  If  the  SMT  solver  returns  sat ,  a  counterexample  to  safety  is  obtained 
using  a  satisfying  assignment.  On  the  other  hand,  if  the  SMT  solver  returns  unsat , 
the  algorithms  utilize  techniques  for  Craig  Interpolation  [43]  to  obtain  assertions 
that  over- approximate  the  reachable  states  at  various  program  locations,  sufficient 
to  show  bounded  safety  (e.g.,  see  [89]).  These  assertions  constitute  a  Hoare-style 
proof  of  bounded  safety.  Now,  if  these  assertions  are  also  invariant  for  the  program, 
i.e.,  hold  for  every  execution  of  the  program,  the  algorithms  terminate.  However, 
it  can  be  very  challenging  in  practice  for  the  assertions  obtained  from  the  proofs  of 
bounded  safety  to  be  invariant,  given  the  undecidability  of  safety. 

To  obtain  better  proofs  of  bounded  safety,  we  describe  an  algorithm  for  automatic 
abstraction  refinement  of  the  input  program,  i.e.,  the  algorithm  tries  to  infer  con¬ 
servative  over-approximations  of  the  transition  relation  sufficient  to  obtain  program 
invariants.  The  key  intuition  is  that  conservative  abstractions  help  us  infer  better 
approximations  of  reachable  states  when  proving  bounded  safety,  that  are  easier  to 

3This  follows  from  the  undecidability  of  safety  in  a  two-counter  machine. 

4This  can  be  checked  using  an  inductive  argument. 


6 


generalize  to  program  invariants.  Moreover,  as  proofs,  in  general,  do  not  depend  on 
all  the  details  of  a  program,  we  can  also  obtain  abstractions  by  hiding  the  details 
irrelevant  for  proofs  of  bounded  safety  (via  Proof-based,  Abstraction  [66,  91]).  Such  an 
abstraction  is  used  to  obtain  a  proof  of  bounded  safety  for  a  bigger  bound  in  the  next 
iteration.  Thus,  our  algorithm  has  a  tight  connection  between  proofs  and  abstrac¬ 
tions.  When  the  abstractions  are  too  coarse,  we  use  spurious  abstract  counterexam¬ 
ples  to  refine  them  (via  Counter  Example- Guided  Abstraction  Refinement  [38]).  This 
approach  is  also  compositional  as  it  tries  to  obtain  an  abstraction  of  the  transition 
relation,  i.e.,  the  data  component  of  the  program,  sufficient  to  show  safety  instead  of 
considering  all  the  details  present  in  the  original  program.  Details  of  our  algorithm 
are  discussed  in  Chapter  3. 


1.2  Safety  of  Probabilistic  Transition  Systems 

Probabilistic  systems  are  increasingly  used  for  the  formal  modeling  and  analysis 
of  a  wide  variety  of  systems  ranging  from  randomized  communication  and  security 
protocols  to  nanoscale  computers  and  biological  processes.  There  exist  algorithms 
for  model  checking  probabilistic  systems  against  temporal  logic  specifications  [18]. 
However,  when  the  systems  are  comprised  of  multiple  parallel  components,  model 
checking  suffers  from  the  state-space  explosion  problem  [39],  where  the  state  space 
of  a  concurrent  system  grows  exponentially  in  the  number  of  its  components. 

The  assume- guarantee  paradigm  for  compositional  reasoning  [98]  addresses  this 
problem  by  separately  verifying  parts  of  the  system  using  assumptions  about  the 
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environment,  without  verifying  the  whole  system  directly.  For  a  system  of  two  com¬ 
ponents,  such  reasoning  is  captured  by  the  following  simple  assume-guarantee  rule. 


1  :  (A)  Li  (P)  2  :  {true)  L2  (A) 

-  (ASym-Gen) 

{true)  Li  ||  L2  (P) 

Here  L\  and  L2  are  system  components,  P  is  a  specification  to  be  satisfied  by 
the  composite  system  {L\  ||  L2)  and  A  is  an  assumption  on  Li’s  environment,  to  be 
discharged  on  L2.  The  challenge  in  using  such  an  assume-guarantee  rule  is  in  coming 
up  with  a  suitable  intermediate  assumption  A  automatically.  Several  other  such  rules 
have  been  proposed,  some  of  them  involving  symmetric  [99]  or  circular  [9,  83,  j 
reasoning.  Despite  its  simplicity,  rule  ASym-Gen  has  been  studied  extensively  in 
the  context  of  non-probabilistic  reasoning  and  is  shown  to  be  effective  in  inferring  a 
suitable  assumption  automatically  [31,  52,  99]. 

When  the  components  Li  and  L2  and  the  specification  P  are  non-probabilistic  and 
finite-state,  there  exist  two  different  kinds  of  algorithms  for  inferring  A.  In  the  first 
kind,  the  algorithms  adapt  known  automata  learning  techniques  (e.g.,  [99])  and  use 
positive  and  negative  examples  from  both  the  premises.  These  are  based  on  the  active 
learning  framework  [14]  where  a  learner  tries  to  learn  an  unknown  system/automaton 
based  on  the  feedback  given  by  a  teacher  in  terms  of  positive  and  negative  examples. 
In  the  assume-guarantee  setting,  the  teacher  is  typically  implemented  by  an  algorithm 
that  checks  the  premises  of  the  rule  and  returns  feedback  correspondingly.  In  the 
second  kind,  the  algorithms  adapt  the  well-known  CEGAR-loop  [38]  to  the  assume- 


guarantee  setting  (e.g.,  [59]).  These  algorithms  only  utilize  negative  examples  (i.e., 
counterexamples) . 

However,  when  the  components  Li  and  L2  and  the  specification  P  are  proba¬ 
bilistic,  efficient  algorithms  are  lacking  for  automating  such  an  assume-guarantee 
framework.  Existing  techniques  target  probabilistic  reachability  properties  and  use 
algorithms  based  on  automata  learning  [51,  52],  However,  even  when  the  problem 
is  decidable  monolithically,  these  algorithms  are  not  guaranteed  to  terminate  due  to 
incompleteness  of  the  inference  rules  [52]  or  due  to  the  undecidability  of  checking  the 
premises  and  of  the  learning  algorithms  used  [51]. 

Given  our  primary  interest  in  safety  properties,  we  describe  algorithms  for  check¬ 
ing  strong  simulation  [102]  between  Labeled  Probabilistic  Transition  Systems  (LPT- 
Ses).  Strong  simulation  is  known  to  preserve  the  weak  safety  fragment  of  probabilis¬ 
tic  CTL  (PCTL),  where  the  bound  on  the  required  probability  of  satisfaction  of  a 
CTL  formula  uses  a  non-strict  inequality  [29],  The  corresponding  instantiation  of 
ASym-Gen  is  shown  below,  where  denotes  the  simulation  conformance  relation 
(see  Chapter  4  for  details)  and  the  components  Li,  L2  and  the  specification  P  are 
all  LPTSes. 


1  :  Lx 


AAP  2  :  L2  A  A 
-  (ASym) 

L\  ||  L2  A  P 
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1.2.1  Counterexamples  to  Strong  Simulation 


One  fundamental  ingredient  of  an  automatic  framework  for  compositional  verifi¬ 
cation  is  the  use  of  counterexamples  (from  failed  simulation  checks)  to  iteratively 
refine  inferred  assumptions.  However,  to  the  best  of  our  knowledge,  the  notion  of 
a  counterexample  has  not  been  previously  formalized  for  strong  simulation  between 
LPTSes.  We  present  a  characterization  of  counterexamples  to  strong  simulation  as 
tree-shaped  LPTSes  and  describe  an  algorithm  to  compute  them. 

1.2.2  Active  Learning  Based  Approach 

Our  first  set  of  algorithms  is  based  on  a  framework  for  active  learning  to  infer  the 
unknown  intermediate  assumptions.  We  develop  the  first  active  learning  framework 
for  inferring  an  unknown  LPTS  (of  minimal  size)  up  to  simulation  equivalence  (2-way 
simulation).  We  also  discuss  decidability  results  for  inferring  intermediate  assump¬ 
tions  in  ASym  using  the  learning  framework.  Details  are  discussed  in  Chapter  5. 

1.2.3  Abstraction- Refinement  Based  Approach 

We  also  propose  an  Assume- Guarantee  Abstraction- Refinement  (AGAR)  algorithm 
to  automatically  build  the  assumptions  used  in  compositional  reasoning.  We  first 
describe  a  CEGAR  [38]  based  algorithm  for  strong  simulation  between  LPTSes, 
which  is  then  adapted  to  the  compositional  setting  to  obtain  AGAR.  Details  are 
discussed  in  Chapter  6. 


10 


1.3  Experimental  Results 


We  have  implemented  the  model  checking  algorithms  described  in  this  thesis  and 
analyzed  the  practical  performance  on  realistic  benchmarks.  The  implementations 
and  benchmarks  are  available  online.5 

The  algorithms  for  software  model  checking  are  implemented  as  part  of  the  tool 
Spacer  (which  stands  for  Software  Proof-based  Abstraction  with  Counter  Example- 
based  Refinement)  for  verifying  C  programs.  The  back-end  is  based  on  the  tool 
Z3  [45]  which  is  used  for  SMT-solving  and  interpolation.  It  supports  propositional 
logic,  linear  rational  arithmetic,  Presburger  arithmetic,  and  bit-vectors  (currently, 
via  bit-blasting).  The  front-end  is  based  on  an  existing  tool  called  UFO  [8]  which 
converts  C  programs  to  the  Horn-SMT  format  of  Z3,  corresponding  to  our  logical 
program  model. 

We  have  a  Java  implementation  of  the  algorithm  AGAR  for  probabilistic  systems. 
We  also  use  the  SMT  solver  Yices  [46]  for  counterexample  generation  and  checking 
strong  simulation,  and  show  experimentally  that  AGAR  can  achieve  significantly 
better  performance  than  monolithic  verification. 


5http : //www. cs . emu . edu/~akomurav/projects. 
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Chapter  2 


SMT-Based  Model  Checking  for 
Recursive  Programs 

2.1  Introduction 

As  mentioned  in  Chapter  1,  several  SMT-based  algorithms  exist  for  verifying  safety 
of  recursive  programs.  Notable  examples  are  Whale  [7],  HSF  [63],  GPDR1  [74], 
Ultimate  Automizer  [68,  69]  and  Duality  [92],  All  of  the  algorithms  are  based  on 
BMC  for  checking  bounded  safety  for  increasing  values  of  the  bound  on  the  call- 
stack  depth.2  The  use  of  BMC  ensures  that  the  algorithms  are  guaranteed  to  find  a 
counterexample  if  the  program  fails  to  satisfy  a  safety  property.  However,  with  the 
exception  of  GPDR,  the  SMT  problems  created  by  these  algorithms  are  monolithic, 
i.e. ,  for  the  entire  program,  and  the  size  of  the  problems  can  grow  quite  large.  In  par- 
^UPDR  stands  for  Generalized  Property  Directed  Reachability. 

2In  this  chapter,  we  assume  that  all  loops  have  been  turned  into  tail  recursive  procedures. 
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ticular,  when  the  SMT  problems  correspond  to  a  bounded  call-stack  depth,  the  size 
grows  exponentially  with  the  bound  in  the  worst-case,  due  to  the  tree-like  unrolling 
of  the  call-graph.  Therefore,  for  the  class  of  Boolean  Programs,  these  algorithms 
are  at  least  worst-case  exponential  in  the  number  of  states.  However,  as  mentioned 
in  Chapter  1,  there  exist  model  checking  algorithms  for  safety  of  Boolean  Programs 
that  are  polynomial  (cubic)  in  the  number  of  states  [101].  The  general  observation 
behind  these  algorithms  is  that  one  can  summarize  the  input-output  behavior  of  a 
procedure,  where  a  summary  of  a  procedure  is  an  input-output  relation  describing 
what  is  currently  known  about  its  behavior.  Thus,  if  a  summary  has  enough  details, 
it  can  be  used  to  analyze  a  procedure  call  without  inlining  [37,  103].  For  a  Boolean 
Program,  the  number  of  states  is  finite  and  hence,  a  summary  can  only  be  updated 
finitely  many  times.  This  observation  led  to  a  number  of  efficient  algorithms  that  are 
polynomial  in  the  number  of  states,  e.g.,  the  analysis  framework  by  Reps,  Horwitz, 
and  Sagiv  (RHS)  [101],  recursive  state  machines  [10],  Bebop  [19]  and  Moped  [19]. 

The  SMT-based  algorithms  mentioned  above  also  utilize  summaries.  If  the  mono¬ 
lithic  SMT  problems  created  are  unsatishable,  the  current  unwinding  of  the  program 
is  insufficient  to  fold  a  counterexample.  In  this  case,  the  algorithms  use  techniques 
based  on  Craig  Interpolation  [43]  to  obtain  over-approximating  summaries  of  pro¬ 
cedures  sufficient  to  show  safety  for  the  current  unwinding.  This  is  repeated  with 
further  unwindings  of  the  program  until  a  counterexample  is  found  or  the  approxi¬ 
mate  summaries  of  the  procedures  are  also  invariant.  However,  as  noted  above,  the 
size  of  the  SMT  problems  created  by  these  algorithms  can  grow  exponentially. 

On  the  other  hand,  the  algorithm  GPDR  [  |  follows  the  approach  of  IC3  [25] 
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by  solving  BMC  incrementally  without  unrolling  the  call-graph.  In  GPDR,  interpo¬ 
lation  is  used  to  obtain  over-approximating  summaries  and  partial  models  denoting 
undesirable  reachable  states  are  cached  for  future.  For  some  configurations  (e.g., 
explicit-state  reasoning),  GPDR  is  worst-case  polynomial  for  Boolean  Programs. 
However,  it  gets  more  challenging  when  the  program  operations  and  formulas  are 
in  a  first-order  language.  In  this  case,  GPDR  might  even  fail  to  find  a  counterex¬ 
ample  despite  the  presence  of  an  SMT  oracle,  unlike  the  guarantee  given  by  other 
BMC-based  algorithms  mentioned  above  (see  Appendix  2. A  for  an  example). 

To  address  the  aforementioned  problems,  we  propose  a  new  SMT-based  algo¬ 
rithm  RecMC  for  analyzing  the  program  compositionally.  That  is,  RecMC  iter¬ 
atively  checks  safety  properties  of  individual  procedures  by  inferring  and  utilizing 
approximating  summaries  of  procedures.  Our  main  insight  is  to  maintain  not  only 
over-approximating  summaries  but  also  under- approximating  summaries  of  the  pro¬ 
cedures.  Syntactically,  our  approximations  are  assertions  over  the  parameters  of 
a  procedure  and  auxiliary  variables  denoting  the  initial  values  of  the  parameters. 
Clarke  showed  that  such  assertions  are  sufficient  to  obtain  a  relatively  complete 
Hoare  proof  system  by  making  use  of  a  Rule  of  Adaptation  [37]. 

We  use  the  terms  may-summary  and  must-summary,  respectively,  to  refer  to  such 
an  over-  and  under- approximation.  While  may-summaries  are  used  to  block  spuri¬ 
ous  counterexamples,  must-summaries  are  used  to  analyze  a  procedure  call  without 
inlining  the  body  of  the  callee.  Thus,  if  the  under-approximations  given  by  the  must- 
summaries  can  be  reused  at  call-sites,  they  help  avoid  redundant  explorations  of  the 
state-space.  However,  the  must-summaries  can  be  too  strong  to  show  falsification 
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and  the  may-summaries  can  be  too  weak  to  show  satisfaction  of  a  bounded  safety 
property.  In  this  case,  our  compositional  algorithm  creates  and  checks  new  bounded 
safety  properties  of  the  callee  procedures  and  updates  the  approximations. 

For  Boolean  Programs,  as  mentioned  previously,  the  number  of  states  is  finite 
and  hence,  the  approximations  can  only  be  updated  finitely  many  times.  As  the 
approximations  are  reused  at  call-sites  in  a  compositional  manner,  RecMC  has  a 
polynomial  time  complexity  for  Boolean  Programs,  by  using  an  argument  similar  to 
that  of  RHS  [101].  Moreover,  assuming  an  SMT  oracle  for  the  first-order  language 
of  the  assertions  and  the  program  operations,  we  show  that  RecMC  terminates  for 
bounded  safety.  To  the  best  of  our  knowledge,  ours  is  the  first  SMT-based  algorithm 
with  such  guarantees. 

Almost  every  step  of  RecMC  introduces  existential  quantifiers  in  the  assertions. 
RecMC  tries  to  eliminate  these  quantified  variables  as,  otherwise,  they  would  accu¬ 
mulate  exponentially  in  the  value  of  the  bound  corresponding  to  the  bounded  safety 
problem.  This  is  because,  if  no  quantified  variable  is  eliminated,  the  compositional 
algorithm  essentially  breaks  down  into  an  algorithm  that  unrolls  the  call-graph  into  a 
tree  where,  as  we  mentioned  earlier,  the  size  of  the  SMT  problems  created  may  grow 
exponentially  in  the  bound  on  the  call-stack.  A  naive  solution  is  to  use  quantifier 
elimination  (QE),  which  results  in  an  equivalent  quant iher- free  formula,  but  which  is 
also  expensive  in  practice.  Instead,  we  develop  an  alternative  approach  that  under¬ 
approximates  QE,  i.e. ,  obtains  a  quant  iher- free  formula  stronger  than  the  original 
formula.  However,  obtaining  arbitrary  under-approximations  can  lead  to  divergence 
of  the  algorithm.  We  introduce  the  concept  of  Model  Based  Projection  (MBP),  for 
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Figure  2.1:  Flow  of  the  algorithm  RecMC  to  check  if  M  \=  ipsafe ■  &o  and  <ju  denote  the  may  and 
must-summary  maps. 

covering  3x  ■  ip(x,y)  by  finitely-many  quantifier-free  under-approximations  obtained 
using  satisfying  models  of  ip(x,y)  (see  Section  2.5).  We  developed  efficient  MBPs  for 
Linear  Rational  Arithmetic  (LRA)  and  Presburger  Arithmetic  (also  known  as  Linear 
Integer  Arithmetic  (LIA))  based  on  the  QE  methods  by  Loos-Weispfenning  [86]  for 
LRA  and  Cooper  [!2|  for  LIA.  We  use  MBP  to  under-approximate  existential  quan¬ 
tification  in  RecMC.  In  the  best  case,  a  partial  under-approximation  suffices  and  a 
complete  quantifier  elimination  can  be  avoided. 


In  summary,  we  present:  (a)  an  efficient,  compositional  SMT-based  algorithm 
for  model  checking  recursive  programs,  that  uses  under-  and  over-approximate  sum¬ 
maries  of  procedure  behavior  (Section  2.4),  (b)  MBP  functions  for  obtaining  quantifier- 
free  under-approximations  of  existential  quantification  for  LRA  and  LIA  (Section  2.5), 

(c)  a  new,  complete  algorithm  for  Boolean  Programs,  with  complexity  polynomial  in 
the  number  of  states,  similar  to  the  best  known  method  [19]  (see  Section  2.4),  and 

(d)  an  implementation  and  an  empirical  evaluation  of  the  approach  (Section  2.6). 
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2.2  Overview 


In  this  section,  we  give  an  overview  of  RecMC  and  illustrate  it  on  an  example. 
Let  V  be  a  recursive  program.  We  assume  that  there  are  no  internal  procedures 
and  that  procedures  cannot  be  passed  as  parameters.  Furthermore,  for  simplicity  of 
presentation,  assume  no  loops,  no  global  variables  and  that  arguments  are  passed 
by  reference.  Let  P(v)  G  V  be  a  procedure  with  parameters  h,  and  let  ho  be  fresh 
variables  not  appearing  in  P  with  |ho|  =  |h|,  denoting  the  initial  values  of  h.  A 
safety  property  for  P  is  an  assertion  p(v 0,h)  in  a  given  assertion  language  with  ho 
and  h  as  free  variables.  We  say  that  P  satisfies  p,  denoted  P(h)  |=  p(v0,v),  iff  the 
Hoare-triple  {h  =  h0}  call  P(y )  {</2(h0,h)}  is  valid.  Note  that  every  Hoare-triple 
corresponds  to  a  safety  property  in  this  sense,  as  shown  by  Clarke  [37]  using  a  Rule 
of  Adaptation. 

An  execution  of  a  procedure  P(y)  is  a  sequence  of  valuations  to  the  variables  in 
scope,  according  to  a  given  underlying  semantics,  beginning  with  an  entry  location 
of  P  and  terminating  with  an  exit  location  of  P.  We  say  that  an  execution  satisfies 
an  assertion  <^(ho,h)  if  assigning  the  initial  and  the  final  valuations  of  the  execution 
to  ho  and  h,  respectively,  makes  p  true.  For  a  natural  number  n  >  0,  we  say  that 
an  execution  uses  a  call-stack  bounded  by  n  if  at  no  point  during  the  execution  there 
are  more  than  n  outstanding  procedure  calls  that  are  not  returned.  We  say  that  p  is 
a  bounded  safety  property  for  P  and  n,  denoted  -P(h)  \=n  p(vo,v),  iff  all  executions 
of  P  using  a  call-stack  bounded  by  n  satisfy  <p. 

The  key  steps  of  RecMC  are  shown  in  Fig.  2.1.  RecMC  decides  safety  for  the 
main  procedure  M  of  V.  RecMC  maintains  two  assertion  maps  cru  and  cr0.  The 
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N  update  au  and  er0 
for  bounds  <  b 


3 


False 

update  au  for  ( P ,  b ) 


True 

update  aQ  for  (P,  b) 


Figure  2.2:  Flow  of  the  algorithm  BndSafety  to  check  P  |=f,  ip. 


must-summary  map  au  maps  each  procedure  P(h)  G  V  to  a  set  of  assertions  over 
tioUtJ  that  under- approximate  its  behavior.  Similarly,  the  may-summary  map  cr0 
maps  a  procedure  P  to  a  set  of  assertions  that  over- approximate  its  behavior.  Given 
P,  the  maps  are  partitioned  according  to  the  bound  on  the  call-stack.  Therefore,  if 
au(P,n),  for  some  n  >  0,  contains  an  assertion  S(vq,v)  with  free  variables  ho  Ur, 
then  6  under-approximates  the  combined  behavior  of  all  executions  of  P  that  use  a 
call-stack  of  depth  at  most  n.  In  other  words,  for  every  model  m  of  5,  there  is  an 
execution  of  P  that  begins  in  m(vo),  the  value  of  ho  under  m,  and  ends  in  m(h),  the 
value  of  h  under  m,  using  a  call-stack  bounded  by  n.  Similarly,  if  S(v0,  h)  G  cr0(P,  n), 
then  6  over-approximates  the  behavior  of  all  executions  of  P  using  a  call-stack  of 
depth  at  most  n,  i.e.,  P(h)  |=n  <5(h0,h). 

RecMC  alternates  between  two  steps:  (A)  deciding  bounded  safety  (that  also 
updates  au  and  aQ  maps)  and  (B)  checking  whether  the  current  proof  of  bounded 
safety  also  proves  unbounded  safety.  It  terminates  when  a  counterexample  or  a  proof 
is  found. 

Bounded  safety,  i.e.,  whether  P  | =&  tp,  is  decided  using  the  algorithm  BndSafety 
shown  in  Fig.  2.2.  Step  1  checks  whether  ip  is  falsified  using  the  current  must- 


19 


T  (t)  { 


M  (m)  { 


if  (t>0)  { 


T  (m); 
D  (m)  ; 
D  (m)  ; 


t  :=  t-2; 
T  (t); 
t  :=  t+1 ; 


D  (d)  { 


} 


d  :=  d-1 ; 


} 


> 


> 


Figure  2.3:  A  recursive  program  with  3  procedures. 


summaries  (eru)  of  the  callees  of  P  at  bound  6  —  1.  If  so,  it  infers  a  new  must-summary 
for  P  at  bound  6  witnessing  the  falsification  of  tp.  Step  2  checks  whether  <p  is  satisfied 
using  the  current  may-summaries  (cr0)  of  the  callees  at  bound  6  —  1.  If  so,  it  infers 
a  new  may-summary  for  P  at  bound  6  witnessing  the  satisfaction  of  ip.  If  the  prior 
two  steps  fail,  there  is  a  potential  counterexample  7r  in  P  where  the  must-summaries 
of  the  callees  are  too  strong  to  witness  n  but  the  may-summaries  are  too  weak  to 
block  it.  Step  3  checks  the  feasibility  of  such  a  path  7r  by  creating  new  bounded 
safety  properties  for  the  callees  of  P  at  bound  6  —  1,  recursively  checking  the  new 
properties,  and  updating  the  assertion  maps. 

We  conclude  this  section  with  an  illustration  of  RecMC  on  the  program  in 
Fig.  2.3  (adapted  from  [37]).  The  program  has  3  procedures:  the  main  procedure  M, 
and  procedures  T  and  D.  The  procedure  M  calls  T  and  D.  The  procedure  T  modifies  its 
argument  t  and  calls  itself  recursively.  The  procedure  D  decrements  its  argument  d. 
Suppose  that  we  want  to  check  if  the  (main  procedure  of  the)  program  satisfies  the 
safety  property  <p  =  mo  >  2m +  4.  The  assertion  maps  au  and  aQ  are  initially  empty. 

In  the  first  iteration  of  RecMC,  the  bound  n  on  the  call-stack  is  0,  i.e. ,  the 
bounded  safety  problem  is  to  check  whether  all  executions  that  do  not  have  any 
procedure  calls  are  safe.  Given  that  the  only  path  in  M  has  procedure  calls,  no 
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M(m)  |=i  mo  >  2 m  +  4? 

iteration  1 

check  new  property:  D(d)  |=o  T? 

iteration  1 

False;  update  au(D ,  0)  with  (d  =  do  —  1) 

iteration  2 

check  new  property:  T(t)  |=o  t0  >  2 i? 

iteration  1 

True;  update  cr0(T,  0)  with  (to  >  2 1) 

iteration  3 

check  new  property:  D(d)  |=0  d  <  d0  —  1? 

iteration  1 

True;  update  a0(D1 0)  with  (d  <  do  —  1) 

iteration  4 

True;  update  <r0(M,  1)  with  (mo  >  2m  +  4) 

Figure  2.4:  A  run  of  BndSafety  for  the  program  in  Fig.  2.3  and  the  bounded  safety  property 
M(m)  )=i  mo  >  2 m  +  4. 


such  executions  exist  and  safety  trivially  holds  for  bound  0.  Fig.  2.4  shows  the 
four  iterations  of  BndSafety  for  the  next  bound  n  =  1,  i.e.,  for  checking  whether 
M{in )  |=i  tp  holds  or  not.  In  the  first  iteration  of  BndSafety,  the  current  may 
and  must-summaries  of  the  callees  are  insufficient  to  satisfy  or  falsify  the  property, 
and  there  is  a  potential  counterexample  along  the  only  path  in  M.  Next,  we  create 
a  new  property  for  a  callee,  by  performing  a  backward  analysis  along  the  potential 
counterexample  path  beginning  with  the  negation  of  the  safety  property,  and  making 
use  of  the  current  summaries  of  the  callees.  In  practice,  one  need  not  be  restricted 
to  a  backward  analysis;  see  Sections  2.4  and  2.6  for  details.  As  shown  in  Fig.  2.4, 
assume  that  a  new  bounded  safety  property  is  created  for  D  and  au (D,0),  the  must- 
summary  map  of  D  at  bound  0,  is  updated  with  a  new  must-summary  that  witnesses 
the  falsification  of  the  property.  In  the  second  iteration  of  BndSafety,  the  current 
summaries  are  still  insufficient  and  assume  that  a  new  property  is  created  for  T  and 
c0(T,  0)  is  updated  with  a  new  may-summary  that  witnesses  the  satisfaction  of  the 
property.  To  create  the  new  property  for  T,  we  make  use  of  the  must-summary  of 
D  computed  in  the  previous  iteration  for  both  the  calls  to  D  in  M.  This  is  where  the 
compositionality  of  the  algorithm  helps  avoid  the  potential  re-computation  of  the 
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must-summary  of  D.  Similarly,  in  the  third  iteration  of  BndSafety,  let  cr0(D,  0)  be 
updated  with  a  new  may-summary.  At  this  point,  the  may-summaries  for  T  and  D 
at  bound  0  are  sufficient  to  establish  bounded  safety  at  n  —  1  in  the  fourth  iteration 
of  BndSafety,  resulting  in  an  update  of  cr0(M,  1). 

Now,  the  may-summary  map  a0  is: 

cr0(M,  1)  =  {m0  >  2m  +  4},  cr0(T,  0)  =  {t0  >  2t},  cr0(D,  0)  =  {d  <  d0  -  1} 

Ignoring  the  bounds,  the  may-summaries  are  invariant.  For  example,  we  can  prove 
that  the  body  of  T  satisfies  f0  >  2 1,  assuming  that  the  calls  do,  i.e., 

{t  =  to}  T [t]  {t0  >  2 1}  h  {t  =  t0}  Body{l)  {t0  >  2 1}, 

where  Body  denotes  the  body  of  a  procedure.  Thus,  step  B  of  RecMC  succeeds  and 
the  algorithm  terminates  declaring  the  program  SAFE. 

In  summary,  RecMC  checks  safety  of  a  recursive  program  in  a  compositional 
manner  by  inferring  under-  and  over-approximations  of  the  behavior  of  procedures. 
We  use  an  SMT-solver  for  automating  the  steps  of  RecMC  and  BndSafety. 


2.3  Preliminaries 

Consider  a  first-order  language  with  equality  and  let  S  be  its  signature,  i.e.,  the  set 
of  non-logical  function  and  predicate  symbols  (including  equality).  An  S-structure 
I  consists  of  a  domain  of  interpretation,  denoted  |/|,  and  assigns  elements  of  |/|  to 
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variables,  and  functions  and  predicates  on  |/|  to  the  symbols  of  5.  Let  p  be  a  formula 
in  the  first-order  language.  We  assume  the  usual  definition  of  satisfaction  of  p  by  /, 
denoted  /  |=  p.  I  is  called  a  model  of  p  iff  /  |=  p  and  this  can  be  extended  to  a  set 
of  formulas.  A  first-order  S -theory  Th  is  a  set  of  5-sentences.  /  satisfies  p  modulo 
Th,  denoted  I  \=th  P,  iff  /  |=  Th  U  { p }.  p  is  valid  modulo  Th ,  denoted  | —Th  iff 
every  model  of  Th  is  also  a  model  of  p. 

Let  /  be  an  5-structure  and  w  be  a  list  of  fresh  function/predicate  symbols  not 
in  5.  A  (5Uw)-structure  J  is  called  an  expansion  of  /  to  w  iff  |  J\  =  |/|  and  J  agrees 
with  /  on  the  assignments  to  all  variables  and  the  symbols  of  5.  We  use  the  notation 
I{w  i — y  u}  to  denote  the  expansion  of  /  to  w  that  assigns  the  function/predicate 
Ui  to  the  symbol  wy.  For  an  5-sentence  p,  we  write  I(p)  to  denote  the  truth  value 
of  p  under  I .  For  a  formula  p(x)  with  a  fixed  ordering  of  the  free  variables  x,  we 
overload  the  notation  I(p)  to  mean  {a  G  |/|lxl  |  I{x  i— >•  a}  |=  p}.  For  simplicity  of 
presentation,  we  sometimes  identify  the  truth  value  true  with  |/|  and  false  with  0. 

We  assume  that  programs  do  not  have  internal  procedures  and  that  procedures 
cannot  be  passed  as  parameters.  Furthermore,  without  loss  of  generality,  we  assume 
that  programs  do  not  have  loops  or  global  variables.  In  the  following,  we  define 
programs  using  a  logical  representation,  as  opposed  to  giving  a  concrete  syntax. 

Definition  1  (Programs  and  Procedures).  A  program  V  is  a  finite  list  of  procedures 
with  a  designated  main  procedure  M  where  the  program  begins.  A  procedure  P  is  a 
tuple  (Zp,op,'Ep,lp,  /3p) ,  where 

1.  Zp,  op,  and  £p  are  disjoint  finite  lists  of  variables  denoting  the  input  values 
of  the  parameters,  the  output  values  of  the  parameters,  and  the  local  variables, 
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respectively, 

2.  £p  is  a  fresh  predicate  symbol  of  arity  \lp\  +  \op\, 

3.  [3p  is  a  quantifier-free  sentence  over  the  signature  (iS>U{Eg  |  Q  G  PjUfpUbpU 
Ip)  denoting  the  body  of  the  procedure,  where  a  predicate  symbol  Eg  appears 
only  positively,  i.e.,  under  even  number  of  negations. 

We  use  Tip  to  denote  Ip  U  op. 

Intuitively,  for  a  procedure  P,  Ep  is  used  to  denote  its  semantics  and  (3p  encodes 
its  body  using  the  predicate  symbol  E q  for  a  call  to  the  procedure  Q.  We  require 
that  a  predicate  symbol  Eq  appears  only  positively  in  (3p  to  ensure  a  fixed-point 
characterization  of  the  semantics  as  shown  later  on.  For  example,  for  the  signature 
S  =  (0,  Succ,  — ,  +,  <,  >,  =),  the  program  in  Fig.  2.3  is  represented  as  (M,  T,  D)  with 
the  main  procedure  M  =  (mo,  m,  Em,  (Io,  h),  /3m),  T  =  (to,  t,  Ep,  (£0,  £{),  (3p),  and 
D  =  (d0,  d,  Ep,  0,  /3d),  where 

(3m  =  Sp(?n0,  A))  A  Sp(t'o,  ^i)  A  Ep(f' i,  m )  (3d  =  (d  =  d0  —  1) 

(2.1) 

(3p  =  (to  —  0  A  to  —  t)  V  (to  >  0  A  £o  =  to  —  2  A  Sp(to,  ^1)  A  t  =  1) 

Here,  we  abbreviate  Succ\ 0)  by  i  and  (m0,to,do)  and  (m,t,d)  denote  the  input 
and  the  output  values  of  the  parameters  of  the  original  program,  respectively.  For  a 
procedure  P,  let  Paths(P)  denote  the  set  of  all  prime-imp licants  of  (3p .  Intuitively, 
each  element  of  Paths (P)  encodes  a  path  in  the  procedure. 

Let  V  =  (Po, . . . ,  Pn)  be  a  program  and  /  be  an  5-structure.  Let  A"  be  a  list 
of  length  n  such  that  each  A f  is  either  (i)  a  truth  value  if  P*  has  no  parameters, 
i.e.,  | vpf\  =  0,  or  (ii)  a  subset  of  tuples  from  if  | vpf\  >  1.  Let  J(J,  X)  denote 
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the  expansion  J{£p0  n-  X0}  . .  .{£Pn  i— >  Xn}.  The  semantics  of  a  procedure  P*  given 
I,  denoted  [Pi]/,  characterizes  all  the  terminating  executions  of  Pj  and  is  defined 
as  follows.  ([Po]/,  •  •  • ,  [P„]/)  is  the  (point-wise)  least  X  such  that  for  all  Q  G  P, 
</(/,  A")  |=  Vuq  U  Jq  ■  (Pq  =>■  T,q(vq)).  This  has  a  well-known  least  fixed-point 
characterization  [37]. 

For  a  natural  number  b  >  0,  denoting  a  bound  on  the  call-stack,  the  bounded 
semantics  of  a  procedure  Pi  given  /,  denoted  [Pi]/,  characterizes  all  the  executions 
using  a  stack  of  depth  bounded  by  b  and  is  defined  by  induction  on  b: 

[PJ5  =  J(/,(0,...,0»(3Vfe), 

I-p.f,  = ./(/,  (I-Pofr1, ....  naiHHSp,  ■  pn),  b  >  o 

Intuitively,  [Pi]?  consists  of  all  input-output  values  of  the  parameters  of  Pi  reach¬ 
able  along  paths  that  do  not  make  any  procedure  calls,  i.e. ,  by  interpreting  every 
predicate  symbol  in  the  body  Ppi  as  0.  Similarly,  [Pi]?,  for  b  >  0,  consists  of 
all  input-output  values  of  the  parameters  reachable  along  paths  that  use  a  stack  of 
depth  bounded  by  b. 

An  environment  is  a  function  that  maps  a  predicate  symbol  Sp  to  a  formula  over 
Vp.  Given  a  formula  r  and  an  environment  E,  we  abuse  the  notation  [•]  and  write 
[r]/5  for  the  formula  obtained  by  instantiating  every  predicate  symbol  Sp  by  P(Ep) 
in  r. 

Let  Th  be  an  5-theory.  A  safety  property  for  a  procedure  P  G  P  is  a  formula 
over  vp.  We  only  consider  safety  properties  that  are  quant iher- free  or  have  one  block 
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of  existential  quantifiers.  P  satisfies  a  safety  property  cp  w.r.t  Th,  denoted  P  \=Th  <P, 
iff  for  all  models  /  of  Th,  [P]/  C  I(tp).  A  safety  property  if  of  the  main  procedure 
M  of  a  program  V  is  also  called  a  safety  property  of  the  program  itself.  Given  a 
safety  property  i/j(vm),  a  safety  proof  for  if  is  an  environment  fl  that  is  both  safe 
and  invariant: 

[=ThlVx  •  Sm(x)  ==>■  i>(x) In  (safety)  (2.2) 

VP  G  V-  |  =Th  [Vhp  U  Ip  ■  (/ 3p  ==>  Sp(hp))]n  (invariance)  (2.3) 

Given  a  safety  property  (p(vp)  and  a  natural  number  b  >  0,  denoting  a  bound  on 
the  call-stack,  a  procedure  P  satisfies  bounded  safety  w.r.t  Th,  denoted  P  \=b,Th  <p, 
iff  for  all  models  I  of  Th,  [P]/  C  I(<p).  In  this  case,  we  also  call  cp  a  may-summary 
for  ( P,b ).  We  call  <p>  a  must-summary  for  (P,  b)  iff  /(</?)  C  [P]j,  for  all  models  /  of 
Th.  Intuitively,  may-summaries  and  must- summaries  for  ( P,b ),  respectively,  over- 
and  under- approximate  [P]J  for  every  model  /  of  Th. 

A  bounded  assertion  map  maps  a  procedure  P  and  a  natural  number  b  >  0  to  a 
set  of  formulas  over  vp.  Given  a  bounded  assertion  map  m  and  b  >  0,  we  define  two 
special  environments  Ufn  and  Obm  as  follows. 

Ubm  :  Sp  ^  \J{5  e  m(P,  b')  |  b'  <b}  Obm  :  SP  ^  /\{S  G  m(P,  b')  \  b'  >  b} 

We  use  Ubx  and  Obm  to  under-  and  over-approximate  the  bounded  semantics.  For 
convenience,  let  I/”1  and  Off  be  environments  that  map  every  symbol  to  _L. 
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RECMCO V,  ip  safe) 

1  TL  i —  0  J  Gu  4 —  0  ;  G  o  i —  0 

2  while  true  do 

3  res,  au,  a0  <-  BndSafety(P,  tpSafe,  n,  au,  aQ) 

4  if  res  is  UNSAFE  then 

5  |_  return  UNSAFE,  au 

6  else 

7  ind ,  a0  <-  CheckInvariance(P,  a0,  n ) 

8  if  ind  then 

9  return  SAFE,  a0 


10 


n  <—  n  +  1 


11 

12 

13 

14 

15 


CheckInvariance^,  Go,  n) 

ind  4—  true 

foreach  P  G  V  do 

foreach  5  G  gq(P,  n)  do 

if  1=  \Pp\o  =>  S  then 
[_  erD  «-  a0  U  ((P,  n  +  1)  i-A  (5) 


16 

17 


else 

j  ind  <—  false 


18 


return  ( ind,a0 ) 


Figure  2.5:  Pseudo-code  of  RecMC. 


2.4  Model  Checking  Recursive  Programs 


In  this  section,  we  present  our  algorithm  RecMC^,  <^sa/e)  for  determining  whether 
a  program  V  satisfies  a  safety  property  (psafe ■  Let  S  be  the  signature  of  the  first-order 
language  under  consideration  and  assume  a  fixed  5-theory  Th.  To  avoid  clutter,  we 
drop  the  subscript  Th  from  the  notation  \=Th  and  \=b,Th-  We  also  show  the  sound¬ 
ness  of  RecMC  and  discuss  its  complexity  guarantees.  An  efficient  instantiation  of 
RecMC  to  Linear  Arithmetic  is  presented  in  Section  2.5. 


27 


Top-level  Loop.  RecMC  maintains  two  bounded  assertion  maps  au  and  <j0  for 
must  and  may-summaries,  respectively.  For  brevity,  for  a  first-order  formula  r,  we 
write  [[t]^  and  [[t]^  to  denote  [r] jjb  and  [[r]0b  ,  respectively,  where  the  environments 
Ubn  and  Obmi  for  a  bounded  assertion  map  m,  are  as  defined  in  Section  2.3.  Intuitively, 
[rjk  and  [t]]),  respectively,  under-  and  over-approximate  r  using  cru  and  aQ. 

The  pseudo-code  of  the  main  loop  of  RecMC  (corresponding  to  the  flow  diagram 
in  Fig.  2.1)  is  shown  in  Fig.  2.5.  RecMC  follows  an  iterative  deepening  strategy. 
In  each  iteration,  BndSafety  (described  below)  checks  whether  all  executions  of  V 
satisfy  ip  safe  for  a  bound  n  >  0  on  the  call-stack,  i.e. ,  if  M  \=n  ip  safe-  BndSafety 
also  updates  the  maps  au  and  a0.  Whenever  BndSafety  returns  UNSAFE ,  the 
must-summaries  in  cru  are  sufficient  to  construct  a  counterexample  to  safety  and 
the  loop  terminates.  Whenever  BndSafety  returns  SAFE ,  the  may-summaries  in 
crQ  are  sufficient  to  prove  the  absence  of  a  counterexample  for  the  current  bound  n 
on  the  call-stack.  In  this  case,  if  aQ  is  also  invariant  (see  (2.3)),  as  determined  by 
CHECKiNVARIANCE,  O"o  is  a  safety  proof  and  the  loop  terminates.  Otherwise,  the 
bound  on  the  call-stack  is  incremented  and  a  new  iteration  of  the  loop  begins.  Note 
that,  as  a  side-effect  of  CheckInvariance,  some  may-summaries  are  propagated 
to  the  bound  n  +  1.  This  is  similar  to  the  push  generalization  phase  in  the  IC3 
algorithm  [25]. 


Bounded  Safety.  We  describe  the  routine  BndSafety^,  ipsafei  n,  cr^lt,  a^mt)  as 
an  abstract  transition  system  [94]  defined  by  the  inference  rules  shown  in  Fig.  2.6. 
Here,  n  is  the  current  bound  on  the  call-stack,  and  a^lt  and  a1™*  are  the  maps  of 
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Init - 

{{M,-npsafe,n)}  ||  v™  ||  a™ 

May _ Q  II  II  cr0  (p,<p,b)  GQ  |=  1(3 pf-1  = =>  ~^p_ 

Q\{{P,rj,c)  |  c<b,  |=  [Sp]o  A  if)  =>  ->77}  ||  au  ||  a0U{(P,b)  hA  i/j} 

where  -0  =  ITp([/3p]o_1,  ~ip) 

MugT  Q  II  °v,  II  cr0  {P,f,b)  €g  tt  g  Paths(P)  [tt]*"1  =»  ^ 

Q\{(P,V,c)  |  C>  b,  y=ip  =>  ->77}  ||  au  U  { (P,  b)  M-  i/j}  ||  a0 


where  ip  =  3£p  ■  { n\bu  1 


Query 


Q  II  CT«  II  0-o  (P,p,b)  S  Q  ^{Ppfu1  =>  -'V3  7 T  £  Paths {P) 

n  =  TTpre  /\^R{a)  ATTsuf  h  I^pre]^1  A  [Sp(a)]|^_1  A  [7TS„/]|^_1  =*  — 

_ V=  I^prelo"1  A  A  jnsuft-1  =>  _ 

Q  U  {(i?,  ip,b—  1)}  ||  au  ||  <70 


where 


4>  =  (3  (vp  U  ip)  \  a  ■  [Pprelo  1  A  |7TSU/1^  1  A  If)  [a  «-  up] 
for  all  {R,  y,  b  —  1)  g  Q,  |=  %p  =>•  -177 


Unsafe  -  -  Gu  ^  ^  ISm1"  =r4>  FsaF 

UNSAFE 


Safe 


0  I!  cru  II  a0 


\=  ptf]"  =>  P  safe 

SAFE 


Figure  2.6:  Rules  defining  BndSafety(7 0,y>safe,n,<TI^lit  ,aIJlit). 


must  and  may-summaries  input  to  the  routine.  A  state  of  BndSafety  is  a  triple 
Q  ||  au  ||  crG,  where  au  and  a0  are  the  current  maps  and  Q  is  a  set  of  triples  (P,  p,  b) 
for  a  procedure  P,  a  formula  p  over  vp,  and  a  number  b  >  0.  A  triple  (P,  p,b)  G  Q 
is  called  a  bounded  reachability  query  and  asks  whether  P  1^  -up,  i.e.,  whether  there 
is  an  execution  in  P  using  a  call-stack  bounded  by  b  where  the  values  of  Up  satisfy  p. 

BndSafety  starts  with  a  single  query  (M,  -up safe,  n)  and  initializes  the  maps  of 
must  and  may-summaries  (rule  Init).  It  checks  whether  M  \—n  psafe  by  generating 
new  queries  as  necessary  (rule  Query)  and  answering  existing  queries  using  existing 
summaries  (rules  May  and  Must),  the  latter  resulting  in  new  summaries.  When 
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there  are  no  queries  left  to  answer,  i.e.,  Q  is  empty,  BndSafety  terminates  with  a 
result  of  either  UNSAFE  or  SAFE  (rules  Unsafe  and  Safe).  We  explain  the  rules 
May,  Must  and  Query  below. 

May  infers  a  new  may-summary  when  a  query  (P,  ip,  b)  can  be  answered  negatively. 
In  this  case,  there  is  an  over- approximation  of  the  bounded  semantics  of  P  at  bound 
b,  obtained  using  the  may-summaries  of  callees  at  bound  b  —  1,  that  is  unsatishable 
with  ip.  That  is,  |=  [/dp]^1  =>■  ~><f-  The  inference  of  the  new  summary  is  by  inter¬ 
polation  |  ]  (denoted  by  Itp  in  the  side-condition  of  the  rule).  Thus,  the  new  may- 

summary  is  a  formula  over  vP  such  that  |=  ([/^p]^1  ==>•  0(hp))  A  (0( vP )  ==>- 
-1  ip).  Note  that  0  over-approximates  the  bounded  semantics  of  P  at  b.  Every  query 
(. P,rj,c )  G  Q  such  that  r)  is  unsatishable  with  the  updated  environment  0^o(T,P)  is 
immediately  answered  and  removed. 

Must  infers  a  new  must-summary  when  a  query  (P,  ip,  b)  can  be  answered  positively. 
In  this  case,  there  is  an  under-approximation  of  the  bounded  semantics  of  P  at  b , 
obtained  using  the  must-summaries  of  callees  at  bound  b—  1,  that  is  satishable  with 
ip.  That  is,  ]/=■  ([/3p]^_1  ==>•  -up.  In  particular,  there  exists  a  path  7r  in  Paths  (P)  such 
that  ]/=■  {nf-1  =>•  -1  ip.  The  new  must-summary  0  is  obtained  by  choosing  such 
a  path  7 r  non-deterministically  and  existentially  quantifying  all  local  variables  from 
Note  that  0  under-approximates  the  bounded  semantics  of  P  at  b.  Every 
query  (P,  r],  c)  G  Q  such  that  7]  is  satishable  with  the  updated  environment  U^u(T,P) 
is  immediately  answered  and  removed. 

QUERY  creates  a  new  query  when  an  existing  query  (P,  ip,  b)  cannot  be  answered  us¬ 
ing  current  summary  maps  au  and  aQ.  In  this  case,  the  current  over-approximation 
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of  the  bounded  semantics  of  P  at  b  is  satisfiable  with  ip  while  its  current  under¬ 
approximation  is  unsatishable  with  ip.  That  is,  \/=  [/3p]o_1  =>•  ~«p  and  |= 

lPPf~l  =>•  -up.  In  particular,  there  exists  a  path  7 r  in  Paths(P)  such  that 
Y -  WS"1  =>•  -i ip  and  |=  [[7r]k_1  =>•  -up.  Intuitively,  7r  is  a  potential  coun¬ 
terexample  path  that  needs  to  be  checked  for  feasibility.  Such  a  path  7 r  is  chosen 
non-deterministically.  tt  is  guaranteed  to  have  a  conjunct  £#(a),  corresponding  to 
a  call  to  some  procedure  R,  such  that  the  under-approximation  [S^(a)]^-1  is  too 
strong  to  witness  an  execution  along  7 r  that  satisfies  ip  but  the  over-approximation 
[S/j(a)]o_1  is  too  weak  to  block  such  an  execution.  That  is,  7 r  can  be  partitioned 
into  a  prefix  7rpre,  a  conjunct  T,p(a)  corresponding  to  a  call  to  R ,  and  a  suffix  7 Tsuf 
such  that  the  following  hold: 


= 

4 

(L 

O  Cr 

1 

> 

11 

(2.4) 

^  Pswr1  = 

=*  ((Krelo  1  A  {iTsuffu  *)  = 

=»  ^P>) 

(2.5) 

Note  that  the  prefix  npre  and  the  suffix  nsuf  are  over-  and  under-approximated,  re¬ 
spectively.  A  new  query  (R,  ijj,b  —  1)  is  created  where  0  is  obtained  by  existentially 
quantifying  all  variables  from  [[7rpre]o_1  A  [tTs^/]^1  A  ip  except  the  arguments  a  of  the 
call,  and  renaming  appropriately.  If  the  new  query  is  answered  negatively  (using 
May),  all  executions  along  7 r  where  the  values  of  Vp  U  Ip  satisfy  [tTs^/J^1  are  spuri¬ 
ous  counterexamples.  An  additional  side-condition  requires  that  0  “does  not  overlap” 
with  r)  for  any  other  query  (R,  rj,  b  —  1)  in  Q.  This  is  necessary  for  termination  of 
BndSafety  (Theorem  2).  In  practice,  the  side-condition  is  trivially  satisfied  by 
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7 Ti 

i  =  1 

St(uIoAo) 

A 

T 

i  =  2 

^d(£ 0,  tl) 

£\  =  to  —  1 

T 

i  =  3 

m  =  £  i  —  l 

T 

Figure  2.7:  Approximations  of  the  only  path  7r  of  the  procedure  M  in  Fig.  2.3. 

always  applying  the  rule  to  (P,  p,  b)  with  the  smallest  b. 

For  example,  consider  the  program  in  Fig.  2.3  represented  by  (2.1)  and  the  query 
(M,p,  1)  where  p  =  m0  <  2m  +  4.  Let  aD  =  0,  au(D}0)  =  {d  =  d0  —  1}  and 
au(T,  0)  =  0.  Let  7 r  =  (Sr(mo,fo)  A  Si)(t'o,^i)  A  E£>(fh,m))  denote  the  only  path  in 
the  procedure  M.  Fig.  2.7  shows  [7Tj]°  and  [7Ti]°  for  each  conjunct  7 q  of  7r.  As  the 
hgure  shows,  |7r]°  is  satishable  with  p,  witnessed  by  the  execution  e  =  (m0  =  3,  £q  = 
3,  l\  —  2,m  =  1).  Note  that  this  execution  also  satisfies  [7t2  A  7t3]°.  But,  [7^]°  is 
too  strong  to  witness  it,  where  is  the  call  Sr(mo,4)-  To  create  a  new  query  for 
T,  we  first  existentially  quantify  all  variables  other  than  the  arguments  m0  and  £0 
from  7 r2  A7T3  A  p,  obtaining  m0  <  2£0.  Renaming  the  arguments  by  the  parameters  of 
T  results  in  the  new  query  (T,  to  <  2£,  0).  Further  iterations  of  BndSafety  would 
answer  this  query  negatively  making  the  execution  e  spurious.  Note  that  this  would 
also  make  all  other  executions  where  the  values  to  satisfy  \ii2  A  7t3]° 

spurious. 

Remark.  Note  that  2.4  above  can  be  equivalently  written  as: 


1=  {hprefo  1  A  [7 Tsuffu  1  A  p)  =>  -4£fl(a)]£  1 
Let  A  =  ([7rpre]|)_1  A  [7^/]^' 1  A  p)  and  B  =  -ijE^a)]*-1.  So,  the  query  created 
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by  the  rule  Query  is  essentially  the  strongest  interpolant  for  A  =>-  B.  One  can 
alternatively  consider  other  interpolants  as  candidates  for  new  queries.  For  example, 
the  weakest  interpolant  B  is  another  candidate  for  the  new  query.  However,  B  is 
independent  of  A  and  is  not  property-driven.  We  leave  these  considerations  for  future 
exploration. 

2.4.1  Soundness  of  BndSafety  and  RecMC 

Soundness  of  RecMC  follows  from  that  of  BndSafety,  which  can  be  shown  by  a 
case  analysis  on  the  inference  rules. 

Theorem  1.  BndSafety  and  RecMC  are  sound. 

Proof.  We  only  show  the  soundness  of  BndSafety;  the  soundness  of  RecMC  easily 
follows.  In  particular,  for  BndSafety(M,  tpsafe,  n,  0, 0)  we  show  the  following: 

1.  if  the  premises  of  UNSAFE  hold,  then  M  tpsafe ,  and 

2.  if  the  premises  of  SAFE  hold,  then  M  |=n  tpsafe- 

It  suffices  to  show  that  the  environments  U%.  and  ,  respectively,  under-  and 
over-approximate  the  bounded  semantics  of  the  procedures,  for  every  0  <  b  <  n. 
In  particular,  we  show  that  the  following  is  an  invariant  of  BndSafety:  for  every 
model  /  of  the  background  theory  Th ,  for  every  procedure  Q  G  V  and  b  G  [0,  n], 

J(f/^(SQ))C[Q]5c/(0^(Sg)).  (2.6) 

Initially,  <ju  and  oa  are  empty  and  the  invariant  holds  trivially.  BndSafety 
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updates  aQ  and  au  in  the  rules  May  and  Must,  respectively.  We  show  that  these 
rules  preserve  (2.6).  We  only  show  the  case  of  MAY.  The  case  of  MUST  is  similar. 

Let  (P,<f,b)  6  Q  be  such  that  May  is  applicable,  i.e. ,  |=  [[/3p]o_1  -up.  Let 
0  =  iTPd^/Sp]^-1, -199).  Note  that  p ,  and  hence  0,  does  not  depend  on  the  local 
variables  ip.  Hence,  we  know  that 

b  (3?P  ■  ISpK"1)  V  (2.7) 

The  case  of  b  =  0  is  easy  and  we  will  skip  it.  Let  /  be  an  arbitrary  model  of  Th. 
Assume  that  (2.6)  holds  at  b  —  1  before  applying  the  rule.  In  particular,  assume  that 
for  all  Q  e  V,  IQ]?-1  C  /(O*;1^)). 

We  will  first  show  that  the  new  may-summary  0  over- approximates  [P]0  Let 
J(I,X )  be  an  expansion  of  /  as  defined  in  Section  2.3. 


ipfi  =  j(i,  mtr\  •  •  • ,  i  Pntr1))^  •  m 

C  J(/,(/(O^1(Ep0)),...,/(O^1(EpJ)))(3£pi  -/3pJ  (hypothesis) 

=  /(  [3£p  •  0p]  Qb-i )  ( O b~1  is  F0- definable) 

=  I{3iP-\pP]0b-i)  (logic) 

=  I(3iP  ■  |[/3p]o_1)  (notation) 

C  J(0)  (from  (2.7)) 


Next,  we  show  that  the  invariant  continues  to  hold.  The  map  of  may-summaries  is 
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updated  to  a'Q  =  a0  U  {(P,  6)  (->•  i/;}.  Now,  <j'0  differs  from  aQ  only  for  the  procedure 
P  and  for  bounds  in  [0,6].  Let  b'  G  [0,6]  be  arbitrary.  Since  (2.6)  was  true  before 
applying  MAY,  we  know  that  [P]j  C  I(Ob^o(T,P)).  As  | [P]f  C  [P]J  C  J('0),  it 
follows  that  iPf;  C  I(0*(EP))  n  /(^)  C  /(O*  (£p)  Aip)  —  I(0*( EP)).  □ 

2.4.2  Termination  and  Complexity  of  BndSafety 

We  will  now  show  that  BndSafety  is  complete  relative  to  an  oracle  for  satisfiability 
of  existentially  quantified  formulas  (i.e. ,  formulas  that  are  quant iher- free  or  have  one 
block  of  existential  quantifiers)  modulo  Th.  Throughout  the  following,  we  assume 
that  such  an  oracle  exists.  Intuitively,  a  must-summary  inferred  by  BndSafety 
corresponds  to  a  path  in  a  procedure  and  given  a  bound  on  the  call-stack,  the  number 
of  such  formulas  is  finite.  This  bounds  the  number  of  may /must-summaries  inferred 
by  BndSafety,  guaranteeing  termination. 

The  following  lemma  shows  that  when  a  query  is  removed  from  Q,  it  is  actually 
answered.  The  proof  is  immediate  from  the  definitions  of  Oba  and  Ub  given  in 
Section  2.3. 

Lemma  1  (Answered  Queries).  Whenever  BndSafety  removes  a  query  from  Q, 
it  is  answered  using  the  known  must  and  may-summaries.  In  particular,  for  every 
query  ( P ,  r),  6)  G  Q  removed  from  Q  by  BndSafety, 

1.  if  the  query  is  removed  by  May,  then  |=  [[Ep][(  -< rj,  and 

2.  if  the  query  is  removed  by  MUST,  then  |^=  [£p]^  -177. 

Next,  we  show  that  current  summaries  are  insufficient  to  answer  existing  queries 
in  Q. 
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Lemma  2  (Pending  Queries).  Q  only  has  the  queries  which  cannot  be  immediately 
answered  by  au  or  a0,  i.e.,  as  long  as  (P,ri,£)  is  in  Q,  the  following  are  invariants 
of  BndSafety. 

1-  V=  PpM  =s-  and 

*•  h  PpK  =►  -v- 

Proof.  We  first  show  that  the  invariants  hold  when  a  query  is  newly  created  by 
Query.  Let  P,  rj  and  £  be,  respectively,  R ,  if[d  •<—  vp]  and  6  —  1,  as  in  the  conclusion 
of  the  rule.  The  last-but-one  premise  of  QUERY  is 

h  l^preto~l  A  P^(a)]^-1  A  [TT^r1  ==►  ^ 


which  implies  that 

h  lSR(a)ln_1  ==►  -  (Krelo-1  A  [? A  (fi)  . 

The  variables  not  in  common,  viz.,  (vp  U  Ip)  \a,  can  be  universally  quantified  from 
the  right  hand  side  resulting  in  |=  [Ep]k_1  =>■  -177.  Similarly,  ==>•  -177 

follows  from  the  last  premise  of  the  rule.  Next,  we  show  that  MAY  and  MUST 
preserve  the  invariants. 

Let  May  answer  a  query  (P,  <p,  £)  with  a  new  may-summary  76  and  let  the  updated 
map  of  may-summaries  be  o'Q  =  cra  U  {(P,  £)  i-A  if}.  Now,  consider  (P,  77,  £')  G  Q  after 
the  application  of  the  rule.  If  i'  >  £,  O^,  =  C)'fTo  and  the  invariant  continues  to  hold. 
So,  assume  £'  <  i.  From  the  conclusion  of  MAY,  we  have  |^=  [Epjf  A  if  =>•  -177. 
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Now,  0%,  (Ep)  =  Ofo(Ep)  A  ip.  So,  the  invariant  continues  to  hold. 

Similarly,  let  MUST  answer  a  query  (P,ip,£)  with  a  new  must-summary  ip  and 
let  the  updated  map  of  must-summaries  be  a'u  =  au  U  {ip  H >  (P,£)}.  Now,  consider 
(P,  r) ,  £!)  G  Q  after  the  application  of  the  rule.  If  P  <  £,  U%,  =  U^u  and  the  invariant 
continues  to  hold.  So,  assume  £'  >  £.  From  the  conclusion  of  Must,  we  have 
|=  ip  =>  -it).  Assuming  the  invariant  holds  before  the  rule  application,  we  also 
have  1=  [Ep]^  ==>-  ->rj.  Therefore,  we  have  |=  [Ep]„  V  ip  =>  ->rj.  Now, 

U%i  (Sp)  =  P5u(Ep)  V  ip.  So,  the  invariant  continues  to  hold.  □ 

The  next  few  lemmas  show  that  the  rules  of  the  algorithm  cannot  be  applied 
indefinitely,  leading  to  a  termination  argument.  Let  N  be  the  number  of  procedures 
in  the  program  P,  p  be  the  maximum  number  of  paths  in  a  procedure,  and  c  be  the 
maximum  number  of  procedure  calls  along  any  path  in  V. 

Lemma  3  (Finitely- many  Must  Summaries).  Given  a  predicate  symbol  E p  and  a 
bound  b,  the  environment  U^u  is  updated  only  0(Nb  ■  pb+1)-many  times. 

Proof.  The  environment  Upu  can  be  updated  for  Ep  and  b  whenever  a  must-summary 
is  inferred  for  P  at  a  bound  b'  <  b.  Now,  a  must-summary  is  obtained  per  path 
(after  eliminating  the  local  variables)  of  a  procedure,  using  the  currently  known 
must-summaries  about  the  callees.  Moreover,  Lemmas  1  and  2  imply  that  no  must- 
summary  is  inferred  twice.  This  is  because  whenever  a  query  is  answered  using  MUST, 
the  query  could  not  have  been  answered  using  already  existing  must-summaries  and 
a  new  must-summary  is  inferred. 

This  gives  the  following  recurrence  Must(b )  for  the  number  of  updates  to  Upu  for 
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a  given  Ep: 


Ip,  6  =  0 

(p  ■  N  +  1)  •  Must(b  —  1),  b  >  0. 

In  words,  for  6  =  0,  the  number  of  updates  is  given  by  the  number  of  must-summaries 
that  can  be  inferred,  which  is  bounded  by  the  number  of  paths  p  in  the  procedure 
P.  For  6  >  0,  the  environment  Uf  is  updated  when  a  must-summary  is  learnt  for 
the  procedure  at  a  bound  smaller  than  or  equal  to  6.  For  the  former,  the  number  of 
updates  is  simply  Must(b  —  1).  For  the  latter,  a  new  must-summary  is  inferred  at 
bound  6  along  a  path  whenever  Ubfl  changes  for  a  callee.  For  N  procedures  and  p 
paths,  this  is  given  by  (p  ■  N  ■  Must{b  —  1)). 

This  gives  us  Must(b )  =  0(Nb  ■  pb+l).  □ 

Lemma  4  (Finitely- many  Queries).  For  (P,tp,b)  G  Q,  Query  is  applicable  only 
0(c  ■  Nb  ■  pb+1)-many  times. 

Proof.  First,  assume  that  the  environments  Ubfl  and  are  fixed.  The  number  of 
possible  queries  that  can  be  created  for  a  given  path  of  P  is  bounded  by  the  number 
of  ways  the  path  can  be  divided  into  a  prefix,  a  procedure  call,  and  a  suffix.  This 
is  bounded  by  c,  the  maximum  number  of  calls  along  the  path.  For  p  paths,  this  is 
bounded  by  c  ■  p. 

Consider  a  path  n  and  its  division,  and  let  a  query  be  created  for  a  callee  R  along 
7 r.  Now,  while  the  query  is  still  in  Q,  updates  to  the  environments  Off1  and  Ub~fl  do 
not  result  in  a  new  query  for  R  for  the  same  division  along  n.  This  is  because,  the 
new  query  would  overlap  with  the  existing  one  and  this  is  disallowed  by  the  second 
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side-condition  of  Query. 

Suppose  that  the  new  query  is  answered  by  MAY.  With  the  updated  map  of  may- 
summary,  the  last  premise  of  QUERY  can  be  shown  to  fail  for  the  current  division 
of  7 r.  If  O'ff1  js  updated,  the  last  premise  continues  to  fail.  So,  a  new  query  can  be 
created  for  the  same  prefix  and  suffix  along  n  only  if  Uff1  is  updated  for  some  callee 
along  Ti.  The  other  possibility  is  that  the  query  is  answered  by  MUST  which  updates 
Ub~l  as  well. 

<->  u 

Thus,  for  a  given  path,  and  a  given  division  of  it  into  prefix  and  suffix,  the  number 
of  queries  that  can  be  created  is  bounded  by  the  number  of  updates  to  Ub~l  which  is 
(N  ■  Must{b  —  1)).  Here,  Must  is  as  in  Lemma  3.  So,  the  number  of  times  Query  is 
applicable  for  a  given  query  (P,ip,b)  is  0(jp-c-N  ■  Must(b— 1)).  As  Must{b )  =  Nb-pb+1, 
we  obtain  the  bound  0(c  •  Nb  ■  pb+1).  □ 

Lemma  5  (Progress).  As  long  as  Q  is  non-empty,  either  May,  Must  or  Query 
is  always  applicable. 

Proof.  First,  we  show  that  for  every  query  in  Q,  either  of  the  three  rules  is  applicable, 
without  the  second  side-condition  in  QUERY.  Let  (P,tp,b)  G  Q.  If  |=  [[/Wo-1  => 
-up,  then  May  is  applicable.  Otherwise,  there  exists  a  path  n  e  Paths (P)  such  that 
Wo  1  is  satisfiable  with  <p,  i.e. ,  Wo-1  -I<A  Now,  if  WW  is  also  satishable 

with  ip,  i.e.,  Y=  W«-1  =>  “'Vb  Must  is  applicable.  Otherwise,  |=  WW  =>■  _,(A 
Note  that  this  can  only  happen  if  b  >  0,  as  otherwise,  there  will  not  be  any  procedure 
calls  along  7r  and  WW  anc^  WW  would  be  equivalent. 

Let  7T  =  7To  A  7Ti  A  . . .  7T/  for  some  finite  l.  Then,  WW  obtained  by  taking  the 
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conjunction  of  the  formulas 


<MMMrV--> 


6-1 


Similarly,  [71]^  1  is  obtained  by  taking  the  conjunction  of  the  formulas 

From  Theorem  1,  we  can  think  of  obtaining  the  latter  sequence  of  formulas  by  con¬ 
joining  ([vri]*— 1  to  [7Tj]o_1  for  every  i.  When  this  is  done  backwards  for  decreasing 
values  of  i,  an  intermediate  sequence  looks  like 


(Mo  \  \IMIu 

As  ([vr] 1  is  unsatishable  with  p,  there  exists  a  maximal  j  such  that  the  conjunction 
of  constraints  in  such  an  intermediate  sequence  are  unsatishable  with  p.  Moreover, 
7 Tj  must  be  a  literal  of  the  form  £j?(a)  as  otherwise,  [[vr^]^1  =  [[TTy]^,- 1  violating 
the  maximality  condition  on  j.  Thus,  all  premises  of  QUERY  hold  and  the  rule  is 
applicable. 

Now,  the  second  side-condition  in  QUERY  can  be  trivially  satisfied  by  always 
choosing  a  query  in  Q  with  the  smallest  bound  for  the  next  rule  to  apply.  This  is 
because,  if  (R,  77,  b  —  1)  is  the  newly  created  query,  there  is  no  other  query  in  Q  for 
R  and  6  —  1.  □ 

Lemmas  4  and  5  imply  that  every  query  in  Q  is  eventually  answered  by  MAY  or 
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Must,  as  shown  below. 

Lemma  6  (Eventual  Answer).  Every  (P,<p,b)  6  Q  is  eventually  answered  by  May 
or  Must,  in  0(b  ■  cb  ■  ( Np applications  of  the  rules. 

Proof.  Firstly,  to  answer  any  given  query  in  Q.  Lemma  4  guarantees  that  the  algo¬ 
rithm  can  only  create  finitely  many  queries.  Lemma  5  guarantees  that  some  rule  is 
always  applicable,  as  long  as  Q  is  non-empty.  Thus,  when  Query  cannot  be  applied 
for  any  query  in  Q,  either  MAY  or  MUST  must  be  applicable  for  some  query.  Thus, 
eventually,  all  queries  are  answered. 

The  total  number  of  rule  applications  to  answer  (P,  <p,  b )  is  then  linear  in  the 
cumulative  number  of  applications  of  QUERY,  which  has  the  following  recurrence: 

f  <9(0),  b  =  0 

m  =  i 

[q(6)(1  +  T(6-1)),  b>  0. 

where  Q(b)  denotes  the  number  of  applications  of  QUERY  for  a  fixed  query  in  Q  at 
bound  b.  From  Lemma  4,  Q(b)  =  0(c  ■  Nb  ■  pb+l).  This  gives  us  T{b)  =  0{b  ■  cb  ■ 
(. Np)° (b2)).  □ 

The  main  termination  theorem  is  an  immediate  consequence  of  the  above  lemma: 
Theorem  2.  Given  an  oracle  for  satisfiability  of  existentially  quantified  formulas 
modulo  Th,  BndSafety("P,  ip,  n,  0, 0)  decides  bounded  safety  infinitely  many  itera¬ 
tions  and  terminates. 

As  an  immediate  corollary,  RecMC  is  guaranteed  to  find  a  counterexample  if 
one  exists. 
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Corollary  1.  RecMC^,  (p)  is  guaranteed  to  return  UNSAFE  with  a  counterexam¬ 
ple  ifV  ^  ip. 

In  contrast,  the  closest  related  algorithm  GPDR  [  I],  mentioned  briefly  in  Sec¬ 
tion  2.1,  does  not  have  such  guarantees.  Finally,  for  Boolean  Programs  RecMC  is  a 
complete  decision  procedure.  Unlike  the  general  case,  the  number  of  reachable  states 
of  a  Boolean  Program,  and  hence  the  number  of  summaries,  is  finite.  Boolean  pro¬ 
grams  are  obtained  when  the  signature  S  is  assumed  to  be  empty,  i.e.,  there  are  no 
non-logical  function  or  predicate  symbols.  Let  N  denote  the  number  of  procedures 
of  a  program  V  and  k  =  max{|tJp|  |  P{vp)  G  V}. 

Theorem  3.  LetV  be  a  Boolean  Program.  Then  RecMC^,  <p)  terminates  in  0(N2- 
2 2k)-many  applications  of  the  rules  in  Fig.  2.6. 

Proof.  First,  assume  a  bound  n  on  the  call-stack.  The  number  of  queries  that  can 
be  created  for  a  procedure  at  any  given  bound  is  0(2k),  the  number  of  possible 
valuations  of  the  parameters  (note  that  QUERY  disallows  overlapping  queries  to  be 
present  simultaneously  in  Q).  For  N  procedures  and  n  possible  values  of  the  bound, 
the  complexity  of  BndSafety^,  tp,  n,  0,  0),  for  a  Boolean  Program,  is  0(N  ■  2k  ■  n). 

Now,  the  total  number  of  may-summaries  that  can  be  inferred  for  a  procedure 
is  also  bounded  by  0(2k).  As  Oko  is  monotonic  in  b ,  the  number  of  iterations  of 
RecMC  is  bounded  by  0(N  ■  2k),  the  cumulative  number  of  states  of  all  procedures. 
Thus,  we  obtain  the  bound  0(N 2  ■  22k)  on  the  number  of  applications  of  the  rules  in 
Fig.  2.6.  □ 

Note  that  the  number  of  states  of  a  Boolean  Program  is  0(N  ■  2k),  so  the  above 
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bound  is  polynomial  in  the  number  of  states.  Moreover,  assuming  that  we  always 
eliminate  Boolean  existential  quantifiers  using  quantifier  elimination,  the  total  com¬ 
plexity  of  RecMC  is  also  polynomial  in  the  number  of  states.  In  contrast,  other 
SMT-based  algorithms,  such  as  WHALE  [7],  are  worst-case  exponential  in  the  num¬ 
ber  of  states  of  a  Boolean  Program.  Also,  note  that  the  complexity  is  quadratic  in 
the  number  of  procedures  as  opposed  to  the  known  upper-bound  which  has  a  lin¬ 
ear  dependency  [19].  This  is  a  manifestation  of  the  iterative  deepening  strategy  of 
RecMC  and  in  particular,  the  may-summaries  computed  by  the  algorithm,  which 
is  necessary  for  handling  programs  over  first-order  theories.  In  contrast,  the  known 
optimal  algorithms  for  Boolean  programs  do  not  compute  may-summaries. 

In  summary,  RecMC  checks  safety  of  a  recursive  program  by  inferring  the  nec¬ 
essary  under-  and  over-approximations  of  procedure  semantics  and  using  them  to 
analyze  procedures  individually. 


2.5  Model  Based  Projection 

The  algorithm  RecMC  described  in  the  previous  section  works  for  an  arbitrary  first- 
order  signature  S  and  a  5-theory  Th  as  long  as  there  is  an  oracle  for  satisfiability 
(of  existentially  quantified  formulas)  modulo  Th.  One  can  also  use  RecMC  as-is  for 
many-sorted  signatures  and  a  corresponding  combination  of  theories,  as  long  as  there 
is  an  SMT  oracle  for  existentially  quantified  formulas  modulo  the  theory  combination. 
In  this  section,  we  restrict  ourselves  to  the  two  combinations  of  Propositional  Logic 
with  the  theories  of  Linear  Rational  Arithmetic  (LRA)  and  Linear  Integer  Arithmetic 
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(LIA)  (also  well  known  as  Presburger  Arithmetic),  and  let  the  corresponding  sorts 
be  Bool,  Rat,  and  Int,  respectively. 

Even  though  RecMC  can  be  used  as-is  for  LRA  and  LIA,  recall  that  BndSafety 
introduces  quantifiers  in  the  formulas  maintained  by  the  algorithm.  This  is  because 
the  may  and  must-summaries  are  formulas  over  the  parameters  of  a  procedure  and 
auxiliary  variables  denoting  their  initial  values,  and  when  creating  a  new  summary, 
all  other  variables  will  be  quantified  away.  Same  is  the  case  with  creating  new 
bounded  safety  properties.  If  these  quantifiers  are  not  eliminated,  every  use  of  a 
summary  at  a  call-site  will  introduce  a  different  copy  of  the  quantified  variables 
which,  in  the  worst-case,  can  end  up  accumulating  exponentially  in  the  bounded 
safety  properties  created  by  the  algorithm.  In  essence,  the  compositional  algorithm 
will  break  down  into  a  non-compositional  one,  similar  to  unrolling  the  call-graph 
into  a  tree  where,  as  we  mentioned  earlier,  the  size  of  the  SMT  problems  created 
can  grow  exponentially  in  the  bound  on  the  call-stack.  On  the  other  hand,  it  is 
expensive  to  use  quantifier  elimination  (QE)  to  obtain  an  equivalent  quantifier- free 
formula.  Instead,  we  propose  an  alternative  approach  that  approximates  QE  with 
quantifier-free  formulas  lazily  and  efficiently. 

In  particular,  we  (a)  introduce  a  model-based  under-approximation  of  QE  for 
existentially  quantified  formulas,  called  Model  Based  Projection  (MBP),  (b)  give 
efficient  (linear  in  the  size  of  formulas  involved)  MBP  procedures  for  Propositional 
Logic,  LRA,  and  LIA,  and  (c)  present  a  modified  version  of  BndSafety  that  uses 
MBP  to  under-approximate  the  existential  quantification  of  variables  out  of  scope, 
and  show  that  it  remains  sound  and  terminating.  Our  MBP  procedures  for  LRA  and 
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LIA  are  based  on  the  QE  algorithms  by  Loos  and  Weispfenning  [86]  and  Cooper  [42], 
respectively. 

Definition  2  (Model  Based  Projection).  Let  y(y)  =  3 x  ■  rjm(x,y)  be  an  existentially 
quantified  formula  where  rjm  is  quantifier  free.  A  function  Projv  from  models  of  7]m 
to  quantifier- free  formulas  over  y  is  a  Model  Based  Projection  (for  rj)  iff 

1.  Projv  has  a  finite  image, 

V  =  Vmk,™  P™jv(M),  and 
3.  for  every  model  M  of  r)m,  M  \=  Proy r]{M) . 

In  other  words,  Projv  covers  the  space  of  all  models  of  rjm(x,y)  by  a  finite  set  of 
quantifier- free  formulas  over  y.  Note  that  there  is  a  trivial  MBP  that  maps  every 
model  of  ym  to  a  quantifier- free  formula  equivalent  to  rj.  However,  when  QE  is 
expensive,  it  is  not  the  most  efficient  MBP  and  our  objective  is  to  obtain  an  MBP 
that  maps  models  to  quantifier-free  under- approximations  of  y.  In  the  following,  we 
describe  MBP  procedures  whose  computation  is  linear  in  time  and  space  given  a 
model. 


2.5.1  MBP  for  Propositional  Logic 

Let  rj(y)  =  3x  ■  rjm(x,y)  be  an  existentially  quantified  formula  where  the  quantified 
variables  in  x  are  all  of  sort  Bool.  Without  loss  of  generality,  assume  that  x  is 
singleton.  Our  MBP  procedure  is  based  on  the  following  equivalence: 

3x  ■  Tjm(x,y)  =  rjm[±)Vrim[T]  (2.8) 
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where  [•]  denotes  a  substitution  for  x. 

We  now  define  an  MBP  BoolProjn  for  Propositional  Logic  as  a  map  from  models 
of  r)m  to  one  of  the  disjuncts  above  depending  on  the  assignment  to  x  in  the  given 
model  M: 

I  Vm  [-L]  j  M\=x  =  ± 

BoolProjv(M)  =  < 

Vm  [T] ,  M  \=  x  =  T 

This  procedure  is  also  used  in  the  GPDR  model  checking  algorithm  [  ]  imple¬ 

mented  in  the  tool  Z3  [15]  and  a  similar  approach  is  used  in  SAT-based  iterative 
quantifier  elimination  in  hardware  verification  [55].  The  following  is  now  immediate. 
Theorem  4.  BoolProj  is  a  Model  Based  Projection. 

2.5.2  MBP  for  Linear  Rational  Arithmetic  (LRA) 

We  begin  with  a  brief  overview  of  Loos-Weispfenning  (LW)  method  [86]  for  quantifier 
elimination  in  LRA.  We  borrow  our  presentation  from  Nipkow  [95]  to  which  we  refer 
the  reader  for  more  details.  Let  r)(y)  =  3x  ■  r)m(x,  y)  as  above,  where  the  variables 
in  x  are  of  sort  Rat.  Let  Th  be  LRA,  or  its  combination  with  Propositional  Logic. 
Without  loss  of  generality,  assume  that  x  is  singleton,  r/m  is  in  Negation  Normal 
Form,  and  x  only  appears  in  the  literals  of  the  form  £  <  x,  x  <  u,  and  x  =  e,  where 
£,  u,  and  e  are  x-free.  Let  lits{rj)  denote  the  literals  of  r/.  The  LW-method  states 
that 

3x-r]m{x,y)  =  j  \J  rjm[e]  V  \J  rjm{£  +  e]  V  r/m[-oo]  |  (2.9) 

\(x=e)£lits(rj)  (£<x)^lits(r])  I 
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where  r]m[-\  denotes  a  virtual  substitution  for  the  literals  containing  x.  Intuitively, 
7jm[e]  covers  the  case  when  a  literal  (x  =  e)  is  true,  j]m[£  +  e]  covers  the  case  where 
£  is  the  largest  lower  bound  satisfied  by  x,  and  ?/m[— oo]  covers  the  remaining  cases. 
We  omit  the  details  of  the  substitution  and  instead  illustrate  it  on  an  example.  Let 
r]m  be  (x  =  e  A  0  1)  \/  {£<  x  A  x  <  u)  \I  (x  <  u  A  02),  where  £,  e,  u,  0  1,  02  are  x-free. 
Then, 

3x  •  rjm  =  rjm[e]  V  rjm[£  +  e]  V  r]m[-oo] 

=  (0i  V(f<eAe<«)  V(e<«A  02)  )v(f<iiV(f<«  A  0  2))  V  0  2 
=  0i  V  (£  <  n)  V  02 


We  now  define  an  MBP  LRAProj,n  for  LRA  as  a  map  from  models  of  rjm  to 
disjuncts  in  (2.9).  Given  M  |=  r)m,  LRAProjv  picks  a  disjunct  that  covers  M  based 
on  values  of  the  literals  of  the  form  x  —  e  and  £  <  x  in  M.  Ties  are  broken  by  a 
syntactic  ordering  on  terms  (e.g.,  when  M  \=  £'  =  £  for  two  literals  £  <  x  and  £'  <  x). 


t]m[e],  if  (x  =  e)  G  lits{r ])  A  M  \=  x  =  e 


LRAProj  V(M)  =  < 


Vm[£  +  e], 


else  if  (£  <  x)  G  lits(r ])  A  M  \=  £  <  x  A 

V(f  <  x)  G  lits(rj)  ■  M  |=  {(£'  <  x)  {£'  <  £)) 


r}m[— 00],  otherwise 


Theorem  5.  LRAProjn  is  a  Model  Based  Projection. 


Proof.  By  definition,  LRAProjv  has  a  finite  image,  as  there  are  only  finitely  many  dis- 
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juncts  in  (2.9).  Thus,  it  suffices  to  show  that  for  every  M  |=  r)m,  M  |=  LRAProjv(M). 

Each  disjunct  in  the  LW  decomposition  (2.9)  is  obtained  by  a  virtual  substitution 
of  the  literals  in  r/m  containing  x.  As  mentioned  in  the  beginning  of  the  section,  we 
assume  that  r/m  is  in  NNF  with  the  only  literals  containing  x  of  the  form  (x  =  e), 
{£  <  x )  or  {x  <  u )  for  x-free  terms  e,  £  and  u.  Let  Subt  denote  the  virtual  substitution 
map  of  literals  when  t  is  either  e,  £  +  e  or  —  oo.  The  LW  method  [86]  defines: 

Sube(x  =  e)  =  T,  Sube{£  <  x)  —  (£  <  e),  Sube(x  <  u)  =  (e  <  u)  (2-10) 

Subi+e{x  =  e)  =  _L,  Subt+e(£'  <  x)  —  {£'  <  £),  Subi+e(x  <  u)  =  {£  <  u)  (2.11) 
Sub_OQ{x  =  e)  =  _L,  Sub-oo^  <  x)  —  _L,  Sub_0 Q(x  <  u)  =  T  (2-12) 

Let  M  |=  r]m  and  LRAProj V(M)  =  rjm[t\  where  t  is  either  e  or  £  +  e  or  —  oo. 
As  r)m  is  in  NNF,  it  suffices  to  show  that  for  every  literal  /i  of  rjrn  containing  x,  the 
following  holds: 

M  h  (R  ==►  ^^(/i))  (2.13) 

We  consider  the  different  possibilities  of  t  below.  For  a  term  v,  let  M\v\  denote 
the  value  of  v  in  M. 

Case  t  =  e.  In  this  case,  we  know  that  M  \=  x  =  e.  Now,  for  a  literal  t  <  x, 

M[£  <  x]  =>  M[£]  <  M[x] 

=  M[£]  <  M[e] 

=  M[£  <  e } 
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M[Subt(£  <  x)] 


{Subt{£  <  x)  =  (£  <  e)}. 


Similarly,  literals  of  the  form  x  <  u  and  x  =  e'  can  be  considered. 

Case  t  —  £  +  e.  In  this  case,  we  know  that  M[l  <  x]  is  true,  i.e. ,  M[£]  <  M[x]  and 
whenever  M {£'  <  x]  is  true,  M[£'  <  £\  is  also  true.  Now,  for  a  literal  £'  <  x, 


M[£'  <x]  =►  M[£'<£] 

=  M[Subt{£'  <  x)] 


{Subt(£'  <x)  =  {£'  <  £)}. 


For  a  literal  x  <  u, 


M[x  <  u]  M[x]  <  M[u\ 
M{£]  <  M[u] 
=>  M[£<u } 

=  M[Subt(x  <  u)] 


{M[£]  <  M\x}} 


{Subt(x  <  u)  —  {£  <  u)} 


For  a  literal  x  =  e,  (2.13)  vacuously  holds  as  M[x  =  e]  is  false. 

Case  t  =  —  oo.  In  this  case,  we  know  that  M\x  =  e }  and  M{£  <  x]  are  false  for 
every  literal  of  the  form  x  =  e  and  £  <  x.  So,  for  such  literals  (2.13)  vacuously 
holds.  For  a  literal  x  <  u,  Subt(x  <  u)  =  T  and  hence,  (2.13)  holds  again. 


□ 
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2.5.3  MBP  for  Linear  Integer  Arithmetic  (LIA) 


We  will  now  present  our  MBP  LIAProjv  for  LIA.  It  is  based  on  Cooper’s  method 
for  Quantifier  Elimination  procedure  for  LIA  [  ].  Let  r](y)  =  3x  ■  r]m(x,y),  where 

is  quantifier  free  and  in  negation  normal  form.  Assume  that  x  is  of  sort  Int 
and  that  Th  is  LIA,  or  its  combination  with  Propositional  Logic.  Without  loss  of 
generality,  let  the  only  literals  containing  x  be  the  form  t  <  x,  x  <  u,  x  =  e  or 
(d  |  ±x  +  ?n),  where  a  \  b  denotes  that  a  divides  6,  the  terms  £,  u,  e  and  w  are  x-free, 
and  d  G  Z  \  {0}.  Let  E  =  {e  \  (x  —  e)  G  lits(r]m)}  be  the  set  of  equality  terms  of  x 
and  L  =  {£  \  (£  <  x)  G  lits{rjm)}  be  the  set  of  lower-bounds  of  x.  Then,  by  Cooper’s 
method, 

(D—l  \  D—l 

V  ^  +  1  +  *]  )  v  V  ^°°[*]-  (2-14) 

i= 0  /  i= 0 

where  D  is  the  least  common  multiple  of  all  the  divisors  in  the  divisibility  literals  of 
r/m ,  [•]  denotes  a  substitution  for  x  and  r/m°°  is  obtained  from  r/m  by  substituting  all 
non-divisibility  literals  as  follows: 

(t  <  x)  v*  T  (x  <  u)  ^  T  (x  =  e)  i-)-  _L  (2.15) 

Intuitively,  the  disjunction  partitions  the  space  of  the  possible  values  of  x.  A 
disjunct  for  (x  =  e)  covers  the  case  when  x  is  equal  to  an  equality  term.  The 
remaining  disjuncts  cover  the  cases  where  l  is  the  maximal  lower  bound  of  x  and 
where  x  satisfies  no  lower  bound.  The  disjunction  over  the  possible  values  of  i  covers 
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the  different  ways  in  which  the  divisibility  literals  can  be  satisfied. 

Model  based  projection  LIAProj  is  defined  as  follows,  conflicts  are  resolved  by 
some  arbitrary,  but  fixed,  syntactic  ordering  on  terms: 


Vm[e\ 


LIAProj  (M)  =  < 


r)m[I  +  1  +  ii\, 


co], 


where  i£  =  M[x  —  [I  +  1)]  mod  D , 


if  x  =  e  G  lits(rj)  A  M  (=  (x  —  e) 

else  if  (£  <  x)  G  lits  (■ rj )  A  M  (=  (£  <  x)  A 

V(f  <  x)  G  lits(r] )  ■  M  \=  ((£'  <  x)  =>  (f  <  £)) 

otherwise 

(2.16) 

i- oo  =  M[x]  mod  D,  and  M[x\  is  the  value  of  x  in 


M. 


The  following  theorem  shows  that  LIAProj  is  indeed  a  model  based  projection. 
The  proof  is  similar  to  that  of  Theorem  5. 

Theorem  6.  LIAProj  is  a  Model  Based  Projection. 


2.5.4  Bounded  Safety  with  MBP 

Given  an  MBP  Pro] r]  for  an  existentially  quantified  formula  rj,  we  have  seen  above 
that  each  quantifier-free  formula  in  the  image  of  Projr/  under-approximates  rj.  As 
above,  we  use  rjm  for  the  quant iher- free  matrix  of  rj.  We  can  now  modify  the  side- 
condition  ip  —  rj  of  MUST  and  Query  in  the  algorithm  BndSafety  to  use  quantifier- 
free  under-approximations  as  follows:  (i)  for  MUST,  the  new  side-condition  is  t/j  = 
Projv(M )  where  M  |=  r]m  A  (p,  and  (ii)  for  QUERY,  the  new  side-condition  is  ip  — 
Projv(M )  where  M  \=  r/m  A  p^a)]^-1.  Note  that  to  avoid  redundant  applications  of 
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the  rules,  we  require  M  to  satisfy  a  formula  stronger  than  7jm.  Intuitively,  (i)  ensures 
that  the  newly  inferred  reachability  fact  answers  the  current  query  and  (ii)  ensures 
that  the  new  query  cannot  be  immediately  answered  by  known  facts.  In  both  cases, 
the  required  model  M  can  be  obtained  as  a  side-effect  of  discharging  the  premises  of 
the  rules.  Soundness  of  BndSafety  is  unaffected  and  termination  of  BndSafety 
follows  from  the  image- finiteness  of  Proj . 

Theorem  7.  Assuming  an  oracle  and  an  MBP  for  Th,  BndSafety  is  sound  and 
terminating  after  modifying  the  rules  as  described  above. 

Proof.  Here,  we  show  that  BndSafety  with  MBP  is  sound  and  terminating. 

First  of  all,  in  presence  of  MBP,  May  is  unaffected  and  a  reachability  fact  inferred 
by  MUST  is  only  strengthened.  Thus,  soundness  of  BndSafety  (Theorem  1)  is 
preserved. 

Then,  it  is  easy  to  show  that  the  modified  side-conditions  to  MUST  and  QUERY 
preserve  Lemmas  1  and  2  and  we  skip  the  proof. 

Then,  we  will  show  that  the  finite-image  property  of  an  MBP  preserves  the 
finiteness  of  the  number  of  reachability  facts  inferred  and  the  number  of  queries 
generated  by  the  algorithm.  Let  d  be  the  size  of  the  image  of  an  MBP.  In  the  proof 
of  Lemma  3,  the  recurrence  relation  will  now  have  an  extra  factor  of  d.  The  rest  of 
the  proof  of  finiteness  of  the  number  of  reachability  facts  remains  the  same.  Similarly, 
in  the  proof  of  Lemma  4,  the  number  of  times  QUERY  can  be  applied  along  a  path 
for  a  fixed  division  and  fixed  environments  O^-1  and  Ubp~l  will  increase  by  a  factor 
of  d.  Again,  the  rest  of  the  proof  of  finiteness  of  the  number  of  queries  generated 
remains  the  same.  That  is,  Lemmas  3  and  4,  and  hence,  Lemma  6,  are  preserved 
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with  scaled  up  complexity  bounds. 

Note  that  Theorem  5  is  unaffected  by  under-approximations. 

Together,  we  have  that  Theorem  2  is  preserved,  with  a  scaled  up  complexity 
bound.  §§[; 

Thus,  BndSafety  with  a  linear-time  MBP  (such  as  LRAProj  )  keeps  the  size  of 
the  formulas  small  by  efficiently  inferring  only  the  necessary  under-approximations 
of  the  quantified  formulas. 

2.6  Implementation  and  Experiments 

We  have  implemented  RecMC  for  analyzing  C  programs  as  part  of  our  tool  Spacer. 
The  back-end  is  based  on  Z3  [45]  which  is  used  for  SMT-solving  and  interpolation. 
It  supports  propositional  logic,  linear  arithmetic,  and  bit- vectors  (via  bit-blasting). 
The  front-end  is  based  on  the  tool  UFO  [8].  It  encodes  safety  of  a  C  program 
by  converting  it  to  the  Horn-SMT  format  of  Z3,  which  corresponds  to  the  logical 
program  representation  described  in  Section  2.3.  Loops  are  handled  by  creating 
fresh  predicate  symbols  denoting  the  loop  invariants  and  encoding  the  corresponding 
verification  conditions.  The  implementation  and  benchmarks  are  available  online3. 
We  evaluated  SPACER  on  three  sets  of  benchmarks: 

(a)  2,908  Boolean  programs  obtained  from  the  SLAM  toolkit,  4 

(b)  1,535  procedural  programs  from  Microsoft’s  SDV  project,  5  and 

3http : //www. cs . emu . edu/~akomurav/projects/ spacer/home .html. 

4https : / / svn . sosy- lab . org/ software/ sv- benchmarks/trunk/ clauses/BOOL/ slam . zip 
5https : / / svn . sosy- lab . org/ software/ sv- benchmarks/trunk/ clauses/ALIA/sdv 
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(c)  797  C  programs  from  the  Software  Verification  Competition  (SV-COMP)  2014  [  ] 


The  numbers  of  programs  mentioned  for  the  second  and  third  sets  of  benchmarks 
above  exclude  programs  with  memory-related  properties  as  SPACER  cannot  handle 
them  yet.  The  797  programs  in  the  third  set  of  benchmarks  also  exclude  programs 
that  can  be  easily  verified  by  our  front-end  (which  converts  a  C  program  to  the 
Horn-SMT  format)  using  common  compiler  optimizations.  Note  that  the  programs 
in  the  last  set  are  not  recursive  and  our  current  front-end  inlines  all  procedure  calls. 
We  call  the  resulting  set  of  encodings  SvCOMP-1.  Note  that  SvCOMP-1  essentially 
corresponds  to  while-programs.  We  introduced  procedural  modularity  in  SVCOMP-1 
by  two  distinct  means:  (a)  factoring  out  maximal  loop-free  fragments  into  new  loop- 
free,  recursion-free  procedures  (the  main  procedure  may  still  have  loops)  to  obtain 
SvCOMP-2,  and  (b)  factoring  out  loops  into  tail-recursive  procedures  (in  an  inside- 
out  fashion  for  nested  loops)  to  obtain  SvCOMP-3.  Our  simple  outlining  procedure 
could  not  handle  some  large  programs  and  SvCOMP-3  has  45  fewer  programs. 

Fig.  2.8  shows  some  characteristics  of  the  Horn-SMT  encodings  for  the  bench¬ 
marks,  when  viewed  according  to  the  logical  program  representation  described  in 
Section  2.3.  In  particular,  for  SVCOMP-1,  we  consider  a  tail-recursive  view  of  the 
inlined  encodings.  The  number  of  calls  along  a  path  roughly  identifies  the  procedu¬ 
ral  modularity  of  the  encodings.  The  number  of  calls  of  a  procedure  (in  the  entire 
program)  identifies  the  potential  number  of  times  a  summary  (may  or  must)  of  the 
procedure  can  be  reused  for  a  given  bound  on  the  call-stack.  Note  that  summaries 
can  also  be  reused  across  different  bounds  on  the  call-stack.  Despite  the  fact  that  the 
average  number  of  procedure  calls  is  low  from  the  figure,  we  can  show  the  practical 
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Figure  2.8:  Some  characteristics  of  the  Horn-SMT  encodings  of  the  benchmarks,  averaged  over  all 
programs  in  the  corresponding  set. 


advantage  of  RecMC  using  these  benchmarks,  as  we  show  below. 

We  compared  Spacer  against  the  implementation  of  GPDR  in  Z3  [  ].  GPDR  is 
inspired  by  the  IC3  hardware  model  checking  algorithm  [25]  and  avoids  unrolling  the 
call-graph.  Thus,  it  creates  and  checks  reachability  queries  for  individual  procedures 
similar  to  RecMC.  However,  it  only  computes  may  summaries  and  because  of  the 
lack  of  must  summaries,  its  query  creation  mechanism  is  quite  different  from  the  rule 
Query.  Moreover,  it  does  not  use  MBP. 

In  our  experiments,  the  resource  limits  were  set  to  30  minutes  of  time  and  16GB 
of  memory,  on  an  Ubuntu  machine  with  a  2.2  GHz  AMD  Opteron(TM)  Processor 
6174  and  516GB  RAM.  Fig.  2.9  and  2.10  show  a  high  level  summary  of  the  results 
in  terms  of  the  number  of  programs  verified  by  SPACER  and  Z3.  Since  there  are 
some  programs  verified  by  only  one  of  the  tools,  the  figures  also  report  the  number 
of  programs  verified  by  at  least  one  tool  in  the  third  row.  We  provide  a  more  detailed 
discussion  of  the  experimental  results  in  the  following.  In  the  scatter  plots  that  are 
shown  below,  a  diamond  indicates  a  time-out  and  a  star  indicates  a  mem-out. 
Boolean  Program  Benchmarks.  Fig.  2.11(a)  shows  the  scatter  plot  of  runtimes 
for  SPACER  and  Z3  for  the  SLAM  benchmarks.  The  runtimes  of  both  the  tools  are 
within  ±5  minutes  for  over  98%  of  the  benchmarks.  Of  the  remaining,  SPACER  is 


55 


Slam 

Sdv 

SAFE 

UNSAFE 

SAFE 

UNSAFE 

Spacer 

1,721 

985 

1,303 

232 

Z3 

1,727 

992 

1,302 

232 

Spacer  or  Z3 

1,727 

992 

1,303 

232 

Figure  2.9:  Number  of  programs  verified  for  Slam  and  Sdv  benchmarks. 
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Figure  2.10:  Number  of  programs  verified  for  SVCOMP  benchmarks. 


better  on  1  benchmark,  Z3  is  better  on  42  benchmarks  which  includes  13  bench¬ 
marks  where  SPACER  runs  out  of  time.  Recall  that  Z3  utilizes  may  summaries 
which  heuristically  avoid  the  possible  exponential  blow-up  associated  with  unwind¬ 
ing  the  call-graph  and  as  the  plot  shows,  such  a  heuristic  approach  can  be  better 
than  SPACER  in  some  cases.  However,  when  we  compared  the  tools  on  the  parametric 
Boolean  program  from  Fig.  1.1,  in  which  the  size  of  the  unrolled  call-tree  necessarily 
grows  exponentially  in  the  number  of  procedures,  SPACER  handles  the  increasing 
complexity  significantly  better  than  Z3,  as  shown  in  Fig.  2.11(b). 

SDV  Benchmarks.  Fig.  2.12  shows  the  scatter  plot  of  runtimes  for  Spacer  and 
Z3  for  the  SDV  benchmarks.  SPACER  clearly  outperforms  Z3  including  a  benchmark 
where  Z3  runs  out  of  time. 

SVCOMP  2014  Benchmarks.  We  begin  with  the  scatter  plot  in  Fig.  2.13(a)  for 
SvCOMP- 1  benchmarks.  As  mentioned  above,  SVCOMP- 1  benchmarks  correspond 
to  while-programs  and  therefore,  do  not  require  must  summaries.  As  the  GPDR 
algorithm  also  computes  may  summaries,  the  plot  in  Fig.  2.13(a)  essentially  shows 
the  advantage  of  using  MBP  in  creating  a  new  query  as  opposed  to  Z3’s  variable 
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Figure  2.11:  Spacer  vs.  Z3  for  (a)  the  Slam  benchmarks  (with  ±5  minute  boundaries),  and  (b) 
the  Boolean  program  in  Fig.  1.1  which  is  parametric  in  the  number  of  procedures. 


Figure  2.12:  Spacer  vs.  Z3  for  the  Sdv  benchmarks. 


substitution  based  on  a  given  model.6 

To  understand  the  effect  of  must  summaries,  we  also  created  a  version  of  SPACER 
that  only  infers  and  utilizes  may  summaries.  We  obtained  this  by  modifying  Z3  to 
use  MBP  in  creating  new  queries.  As  shown  in  Fig.  2.13(b),  the  advantage  of  using 
must  summaries  is  quite  significant  on  SVCOMP-2  benchmarks. 

So,  a  combination  of  MBP  and  must  summaries  is  expected  to  result  in  significant 
improvements  over  using  may  summaries  alone.  This  is  shown  experimentally  in 
Fig.  2.14(a)  and  2.14(b)  for  the  SVCOMP-2  and  SvCOMP-3  benchmarks  which  show 

6Z3  first  tries  to  eliminate  existential  quantifiers  by  using  equalities  with  ground  terms  present 
in  the  input  formula  and  resorts  to  model  substitution  otherwise. 
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Z3  (secs) 


SPACER-May  Only  (secs) 


(a) 


(b) 


Figure  2.13:  The  advantage  of  (a)  MBP,  over  SvcOMP-1,  and  (b)  must  summaries,  over  Svcomp-2, 
in  Spacer.  For  SVCOMP-1,  must  summaries  are  not  required  and  MBP  is  the  only  key  difference 
between  Spacer  and  Z3. 


Z3  (secs)  Z3  (secs) 

(a)  (b) 


Figure  2.14:  Spacer  vs.  Z3  for  the  benchmarks  (a)  Svcomp-2  and  (b)  Svcomp-3. 


that  SPACER  is  significantly  better  than  Z3  on  most  of  the  programs. 

Recall  that  the  rule  Query  checks  the  feasibility  of  a  potential  counterexample 
path  7 r  by  recursively  creating  a  new  reachability  query  for  a  procedure  R  called  along 
7 r.  Due  to  our  logical  representation  of  a  program,  one  can  consider  an  arbitrary 
permutation  of  the  conjuncts  of  n  when  applying  the  rule  and  the  choice  of  the 
procedure  R  is  not  deterministic.  Our  current  implementation  in  SPACER  can  order 
the  conjuncts  either  in  the  given  order  or  in  the  reversed  order  and  for  lack  of  good 
heuristics,  we  do  not  consider  other  permutations.  These  two  orderings  correspond 
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SPACER-Eager  (secs)  Z3-Lazy  (secs) 

(a)  (b) 

Figure  2.15:  Effect  of  the  order  of  query  creation  in  Query  in  Spacer  and  the  corresponding  order 
of  query  handling  in  Z3,  on  Svcomp-2  benchmarks. 


to  top-down  and  bottom-up  feasibility  analyses.  In  particular,  the  plot  shown  in 
Fig.  2.14(a)  corresponds  to  a  bottom-up  analysis. 


As  mentioned  in  the  beginning,  the  SvCOMP-2  benchmarks  are  obtained  by  tak¬ 
ing  the  while-program  encodings  in  SvCOMP-1  and  factoring  out  maximal  loop- 
free  fragments  into  new  loop-free,  recursion- free  procedures.  Furthermore,  as  also 
mentioned  in  the  beginning,  loops  are  encoded  in  SVCOMP-1  by  introducing  new 
predicate  symbols  that  denote  loop  invariants  and  by  encoding  the  corresponding 
verification  conditions.  So,  a  path  in  a  procedure  in  the  resulting  logical  encoding 
(see  Section  2.3)  contains  at  most  two  calls,  one  corresponding  to  an  invariant  at 
a  control  location  and  the  other  corresponding  to  a  newly  introduced  procedure  for 
a  loop-free  fragment.  Thus,  a  top-down  analysis  refines  the  may  summaries  of  the 
new  procedures  only  when  necessary,  similar  to  a  CEGAR-style  reasoning  where  the 
may  summaries  of  the  new  procedures  abstract  the  loop-free  fragments.  We  call 
this  a  lazy  refinement  strategy.  In  contrast,  a  bottom-up  analysis  on  these  bench¬ 
marks  corresponds  to  an  eager  refinement  strategy  which  is  shown  in  Fig.  2.14(a). 
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Svcomp-1  (secs)  Svcomp-1  (secs) 

(a)  (b) 

Figure  2.16:  Comparison  of  Spacer’s  behavior  on  the  various  encodings  of  SVCOMP  benchmarks. 
For  the  plot  in  (a),  we  use  Spacer  in  the  lazy  mode  for  Svcomp-2. 


Fig.  2.15(a)  shows  a  scatter  plot  of  runtimes  on  SvCOMP-2  comparing  the  behavior 
of  SPACER  for  the  two  orderings.  While  it  is  unclear  from  the  figure  which  ordering 
is  better,  SPACER  continues  to  outperform  Z3  even  with  the  lazy  strategy,  as  shown 
in  Fig.  2.15(b). 


Finally,  as  an  interesting  exercise,  we  compared  the  behavior  of  SPACER  on  var¬ 
ious  encodings  of  the  SVCOMP  benchmarks.  For  the  comparison  of  runtimes  be¬ 
tween  Svcomp-2  and  Svcomp-1,  we  considered  the  lazy  mode  of  Spacer  for  the 
Svcomp-2  encodings,  which  essentially  corresponds  to  abstract  reasoning  by  infer¬ 
ring  sufficient  summaries  of  the  loop-free  fragments.  Fig.  2.16(a)  shows  the  runtime 
comparison  for  the  benchmarks.  While  the  SVCOMP-2  encodings  seem  to  be  worse 
overall,  the  difference  in  performance  between  the  encodings  is  less  clear  when  we 
restrict  ourselves  to  the  harder  benchmarks,  e.g.,  where  the  SVCOMP-1  encodings 
need  more  than  5  minutes  of  runtime.  However,  as  we  will  see  in  Chapter  3,  abstrac¬ 
tion  can  be  quite  powerful  and  we  plan  to  incorporate  the  ideas  from  that  chapter 
into  the  framework  of  RecMC  in  the  future.  Then,  Fig.  2.16(b)  shows  the  runtime 
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comparison  for  the  encoding  SVCOMP-3  against  SvCOMP-1.  Recall  that  SvCOMP-3 
encodings  are  obtained  by  factoring  out  loops  into  tail-recursive  procedures.  In  other 
words,  we  are  replacing  the  inference  of  loop  invariants  by  that  of  summaries  of  the 
corresponding  tail-recursive  procedures.  Whereas  a  loop  invariant  depends  on  the 
variables  in  scope,  the  signature  of  the  corresponding  tail-recursive  procedure,  and 
hence  its  summary,  depends  on  two  copies  of  the  variables  in  scope  which  denote 
their  values  before  and  after  a  loop  iteration.  As  the  plot  shows,  this  can  negatively 
affect  the  performance  of  verification. 

Overall,  we  have  shown  significant  practical  benefits  of  the  core  ideas  behind 
RecMC  using  our  implementation  in  SPACER  and  various  realistic  benchmarks. 


2.7  Related  Work 

There  is  a  large  body  of  work  on  interprocedural  program  analysis.  It  was  pointed  out 
early  on  that  safety  verification  of  recursive  programs  is  reducible  to  the  computation 
of  a  fixed-point  over  relations  representing  the  input-output  behavior  of  each  proce¬ 
dure  [37].  The  term  summary  is  used  for  such  a  relation  in  the  functional  approach  of 
Sharir  and  Pnueli  [103].  Reps,  Horwitz,  and  Sagiv  [101]  showed  that  for  a  large  class 
of  finite,  interprocedural  dataflow  problems,  the  summaries  can  be  computed  in  time 
polynomial  in  the  number  of  dataflow  facts  and  procedures.  Ball  and  Rajamani  [  | 

adapted  the  RHS  algorithm  to  the  verification  of  Boolean  Programs  as  part  of  the 
SLAM  project  for  software  model  checking  using  a  CEGAR-style  loop  with  predicate 
abstraction  [62],  Following  SLAM,  other  software  model  checkers,  e.g.,  BLAST  [  ] 
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and  MAGIC  [30],  also  implemented  predicate  abstraction  based  algorithms.  While 
predicate  abstraction  is  used  to  obtain  over-approximations  of  procedure  semantics, 
these  approaches  do  not  use  under-approximations  as  we  do. 

In  the  context  of  predicate  abstraction,  the  algorithm  SMASH  also  combines  over- 
and  under-approximations  for  analyzing  procedural  programs  [60].  However,  the 
summaries  in  SMASH  can  have  auxiliary  variables  which  differ  from  one  calling  con¬ 
text  to  another,  restricting  the  reusability  of  the  summaries.  SMASH  also  under¬ 
approximates  existential  quantification  in  computing  the  results  of  the  post  and  pre 
operations,  but  unlike  RecMC,  the  under-approximations  are  obtained  using  con¬ 
crete  values  encountered  during  testing  of  the  program. 

As  mentioned  earlier  in  the  paper,  several  SMT-based  algorithms  have  been  pro¬ 
posed  for  safety  verification  of  recursive  programs,  including  Whale  [7],  HSF  [63], 
Duality  [92],  Ultimate  Automizer  [68,  69],  and  Corral  [84].  These  algorithms  share 
a  similar  structure  -  they  use  SMT-solvers  to  look  for  counterexamples  and  interpo¬ 
lation  to  compute  over-approximating  procedure  summaries.  The  algorithms  differ 
in  the  SMT  encoding  and  the  heuristics  used.  However,  in  the  worst-case,  they 
completely  unroll  the  call  graph  into  a  tree. 

The  work  closest  to  ours  is  GPDR  [  1],  which  extends  the  hardware  model  check¬ 
ing  algorithm  IC3  of  Bradley  [25]  to  SMT-supported  theories  and  recursive  pro¬ 
grams.  Unlike  RecMC,  GPDR  does  not  maintain  must-summaries.  In  the  context 
of  Fig.  2.6,  this  means  that  au  is  always  empty  and  there  is  no  MUST  rule.  Instead, 
the  Query  rule  is  modified  to  use  a  model  M  that  satisfies  the  premises  (instead 
of  our  use  of  the  entire  path  n  when  creating  a  query).  Furthermore,  undesirable 
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reachable  states  are  cached.  While  the  algorithm  terminates  for  Boolean  programs, 
a  formula  can  have  infinitely  many  models  in  the  general  case  of  first-order  languages 
and  GPDR  might  end  up  applying  the  Query  rule  indefinitely  (see  Appendix  2. A). 
In  contrast,  RecMC  creates  only  finitely  many  queries  for  a  given  bound  on  the 
call-stack  depth  and  is  guaranteed  to  find  a  counterexample  if  one  exists. 

In  the  context  of  Boolean  programs,  there  also  exists  a  SAT-based  summarization 
technique  that  allows  extra  choice  variables  in  the  formulas  and  thereby  requires  a 
Quantified  Boolean  Formulas  (QBF)  solver  to  check  for  convergence  [20]. 


2.8  Conclusion 

We  presented  RecMC,  a  new  SMT-based  algorithm  for  model  checking  safety  prop¬ 
erties  of  recursive  programs.  For  programs  and  properties  over  decidable  theories, 
RecMC  is  guaranteed  to  find  a  counterexample  if  one  exists.  To  our  knowledge, 
this  is  the  first  SMT-based  algorithm  with  such  a  guarantee  while  being  polyno¬ 
mial  for  Boolean  Programs.  The  key  idea  is  to  use  a  combination  of  under-  and 
over-approximations  of  the  semantics  of  procedures,  avoiding  re-exploration  of  parts 
of  the  state-space.  We  described  an  efficient  instantiation  of  RecMC  for  Linear 
Arithmetic  (over  rationals  and  integers)  by  introducing  Model  Based  Projection  to 
under-approximate  the  expensive  quantifier  elimination.  We  have  implemented  it  in 
our  tool  SPACER  and  shown  empirical  evidence  that  it  significantly  improves  on  the 
state-of-the-art. 

In  the  future,  we  would  like  to  explore  extensions  to  other  theories.  Of  particular 
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interest  are  the  theory  EUF  of  uninterpreted  functions  with  equality  and  the  theory 
of  arrays.  The  challenge  is  to  deal  with  the  lack  of  quantifier  elimination.  Another 
direction  of  interest  is  to  combine  RecMC  with  Proof-based  Abstraction  [66,  80,  91], 
which  also  forms  the  basis  of  the  next  chapter,  to  explore  a  combination  of  the 
approximations  of  procedure  semantics  with  transition-relation  abstraction. 

The  algorithm  RecMC  and  the  results  presented  in  this  chapter  are  published 
as  part  of  the  proceedings  of  CAV  2014  [81]. 
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2. A  Divergence  of  GPDR  for  Bounded  Call-Stack 


Consider  the  program  {{M,  L,G),  M)  with  procedures  M  =  (yo,  y,^M,  (x,  n),  /3m), 
L  =  (n,  (x,y,i),  SL,  (x0,  y0,  i0),  0L),  and  G  =  {x0,  xlt  SG,  0,  Pg)  where: 

Pm  =  T,L(x,y0,n,n)  A  EG(x,y)  A  n  >  0 
/3l  =  (*  =  0Ax  =  0A?/  =  0)V 

(El(x o,  yo,  io,  n)  A  x  =  x0  +  1  A  y  =  y0  +  1  A  i  =  i0  +  1  A  i  >  0) 

(3g  =  (x  =  xo  +  1) 

The  GPDR  [  |  algorithm  can  be  shown  to  diverge  when  checking  the  bounded 

safety  problem  M  \=2  yo  <  y,  for  e.g.,  by  inferring  the  diverging  sequence  of  over¬ 
approximations  of  [L]1:  (x  <  2  =>■  y  <  1),  (x  <  3  =>■  y  <  2), . . . . 

We  also  observed  this  behavior  experimentally  (Z3  revision  d548c51  at 
http :  //z3 .  codeplex .  com).7 


7Horn-SMT  file:  http://www  . cs . emu. edu/~akomurav/projects/spacer/gpdr_diverging. srat2. 
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Chapter  3 


Abstraction  in  SMT-Based 
Model  Checking 

3.1  Introduction 

As  described  in  Chapter  1,  SMT-based  model  checkers  work  by  deciding  bounded 
safety  for  increasing  values  of  the  bound  on  the  length  of  an  execution.  When  the 
safety  property  holds,  the  termination  of  such  algorithms  in  practice  depends  on 
whether  a  proof  of  bounded  safety  can  be  found  that  also  proves  (unbounded)  safety. 
Not  surprisingly,  given  the  undecidability  of  safety,  this  can  be  quite  challenging 
to  achieve  in  practice.  In  this  chapter,  we  present  Spacer1,  an  algorithm  that 
incorporates  automatic  abstraction  refinement  into  SMT-based  model  checking. 
Consider  the  safe  program  Pg  (adapted  from  [65])  shown  in  Fig.  3.1.  Here, 

1  Software  Proof-based  Abstraction  with  CounterExample-based  Refinement. 
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0 :  x=0 ;  y=0 ;  z=0 ;  w=0 ; 

1:  while  (nd_bool())  { 

2:  if  (nd_bool())  {x++;  y=y+100;} 

3:  else  if  (nd_bool () ) 

4:  if  (x>=4)  {x++ ;  y++;> 

5:  else  if  (y>10*w  &&  z>=100*x)  {y=-y;} 

6:  t=l; 

7:  w=w+t ;  z=z+(10*t); 

> 

8:  assert (! (x>=4  &&  y<=2)); 

Figure  3.1:  A  program  Pg  adapted  from  an  example  by  Gulavani  et  al.  [65]. 

nd_bool  is  a  routine  that  returns  a  Boolean  value.2  Pg  is  hard  for  existing  SMT- 
based  algorithms.  For  example,  the  implementation  of  the  algorithm  GPDR3  [  |  in 

Z3  [45]  (v4.3.1)  cannot  verify  the  program  in  an  hour.  However,  an  abstraction  of  the 
program,  Pg,  obtained  by  replacing  line  6  with  a  non-deterministic  assignment  to  t 
is  verified  by  the  same  tool  in  under  a  second.  Our  implementation  of  Spacer  Ends 
a  safe  abstraction  of  Pg  in  under  a  minute  (the  transition  relation  of  the  abstraction 
we  automatically  computed  is  a  non-trivial  generalization  of  that  of  Pg  and  does  not 
correspond  to  Pg). 

The  key  intuition  behind  SPACER  is  that  a  good  abstraction  of  the  program  can 
lead  to  a  good  proof  of  bounded  safety.  That  is,  the  assertions  in  a  proof  of  bounded 
safety  that  over-approximate  the  reachable  states  at  the  top  of  a  loop  or  the  behavior 
of  a  procedure  can  be  less  dependent  on  the  bound,  because  of  the  abstraction.  This 
can,  in  turn,  help  in  faster  convergence  to  inferring  invariants  in  a  fewer  number 
of  iterations  of  bounded  safety.  As  a  proof  does  not  utilize  all  the  details  of  the 
program,  in  general,  SPACER  obtains  a  program  abstraction  by  hiding  the  details  of 

2In  other  words,  assume  that  the  behavior  of  nd_bool  is  unknown.  So,  for  the  purpose  of 
verification,  nd_bool  effectively  returns  either  true  or  false  non-deterministically. 

3GPDR  stands  for  Generalized  Property  Directed  Reachability. 
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Program  P 


SAFE  UNSAFE 

Figure  3.2:  An  overview  of  Spacer. 


the  transition  relation  irrelevant  for  a  proof  of  bounded  safety  (called  Proof-Based 
Abstraction  [66,  91]  (PBA)).  However,  an  abstraction  can  be  too  coarse  in  which  case 
we  utilize  spurious  abstract  counterexamples  for  refinement  (called  Counter  Example 
Guided  Abstraction  Refinement  [38]  (CEGAR)). 

Fig.  3.2  shows  the  high  level  flow  of  our  algorithm  SPACER.  We  assume  that  the 
input  program  P  is  annotated  with  the  given  safety  property  (e.g.,  using  assert 
statements).  Spacer  begins  with  an  initial  abstraction  A  of  P  (which  can  be  P 
itself).  Each  iteration  of  SPACER  starts  by  obtaining  an  under-approximation  U  of  A 
and  checking  safety  of  U  (steps  1  and  2).  The  under-approximations  we  will  consider 
in  this  chapter  are  obtained  by  bounding  the  length  of  an  execution.  If  U  is  safe, 
we  obtain  a  proof  7 Tjj  (as  invariants),  and  otherwise,  we  obtain  a  counterexample 
to  safety  c€\j.  In  practice,  the  safety  check  is  implemented  using  an  interpolating 
SMT-solver  (e.g.,  [45,  64,  7  ])  or  a  generalized  Horn-Clause  solver  (e.g.,  [63,  74,  92], 
including  the  algorithm  RecMC  described  in  Chapter  2).  If  U  is  proved  safe,  it  is 
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first  checked  whether  the  formulas  in  nu  are  also  invariant  for  the  original  program 
P,  in  which  case  Spacer,  outputs  SAFE  (step  3);  otherwise,  a  new  abstraction  of  P 
is  obtained  using  the  proof  -Ku  (step  4;  see  below  for  details)  and  the  next  iteration 
begins.  On  the  other  hand,  if  U  is  proved  unsafe,  is  an  abstract  counterexample 
and  needs  to  be  checked  for  feasibility  in  P  (step  5;  this  is  based  on  the  well-known 
CEGAR  approach  [38]).  If  is  feasible,  Spacer  outputs  UNSAFE;  otherwise,  the 
abstraction  A  is  refined  to  eliminate  the  spurious  counterexample  (step  6)  and  the 
next  iteration  begins.  Spacer  is  described  in  Section  3.4  and  a  detailed  run  of  the 
algorithm  on  an  example  is  given  in  Section  3.2. 

Note  that  the  left  iteration  of  SPACER  (steps  1-4)  is  PBA:  in  each  iteration,  an 
under-approximation  is  verified,  a  new  abstraction  based  on  the  proof  is  computed 
and  a  new  under-approximation  is  constructed.  To  the  best  of  our  knowledge,  this  is 
the  first  application  of  PBA  to  Software  Model  Checking.  The  right  iteration  (steps 
1,  2,  5,  6)  is  CEGAR:  in  each  iteration,  (an  under-approximation  of)  an  abstraction 
is  verified  and  refined  by  eliminating  spurious  counterexamples.  SPACER  exploits  the 
natural  duality  between  the  two. 

We  have  implemented  Spacer  using  the  GPDR  engine  inside  the  tool  Z3  [  ] 
as  SOLVE  (see  Section  3.5)  and  evaluated  it  on  many  benchmarks  from  the  2nd 
Software  Verification  Competition1  (SV-COMP’13).  Our  experimental  results  (see 
Section  3.6)  show  that  abstraction  significantly  improves  the  performance  of  SMT- 
based  model  checking  on  hard  benchmarks. 

In  summary,  we  present:  (a)  a  new  algorithm  SPACER  that  combines  abstraction 

4  http : / / sv- comp . sosy- lab . org 
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with  SMT-based  model  checking  and  tightly  connects  proof-  and  counterexample- 
based  abstraction-refinement,  (b)  an  implementation  of  Spacer  using  Z3,  and  (c) 
experimental  results  showing  the  effectiveness  of  SPACER. 


3.2  Overview 

In  this  chapter,  we  restrict  ourselves  to  programs  that  can  be  represented  by  tran¬ 
sition  systems.5  Let  P  be  a  program  represented  by  the  transition  system  (v,  t(v), 
t(v,v'),  err(v)),  where  v  is  the  list  of  state  variables  and  t,  r,  and  err  denote  the  ini¬ 
tial  condition,  the  transition  relation,  and  the  error  condition,  respectively.  Note  that 
we  use  primed  variables  to  denote  the  next-state  values.  Below,  we  give  a  brief  expla¬ 
nation  of  our  abstraction  mechanism  based  on  the  proofs  (of  under- approximations) 
obtained  in  an  iteration  of  SPACER. 

Proof-Based  Abstraction.  Given  P  and  a  proof  tt  of  a  property  of  P  (for  e.g., 
7T  is  a  proof  of  bounded  safety  of  P  for  some  bound  on  the  possible  executions),  the 
goal  of  Proof-Based  Abstraction  (PBA)  is  to  obtain  a  program  P  such  that 

1.  P  is  an  abstraction  of  P,  i.e.,  P  ^  P,  and 

2.  7 r  proves  the  property  for  P. 

Here,  denotes  the  usual  simulation  conformance  between  transition  systems.  Intu¬ 
itively,  when  7T  proves  safety  of  P  for  a  given  bound  b  on  the  length  of  an  execution, 
such  a  proof  preserving  abstraction  mechanism  ensures  that  tt  continues  to  prove 

5In  particular,  this  disallows  procedure  calls.  In  principle,  the  ideas  presented  here  can  be 
combined  with  those  in  Chapter  2  to  handle  procedural  programs  as  well,  but  we  leave  it  for  future 
exploration. 
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bounded  safety  for  P.  Thus,  for  the  future  iterations  of  Spacer,  we  can  use  P, 
instead  of  P,  with  the  hope  of  getting  proofs  of  bounded  safety  that  depend  less  on 
the  bound  and  can  lead  to  faster  convergence  to  inferring  invariants.  When  the  ab¬ 
straction  is  too  coarse,  we  use  the  well-known  CEGAR  approach  [38]  for  refinement. 

Our  notion  of  proof  in  PBA  is  slightly  different  from  a  refutation  proof  given 
by  a  SAT  solver  used  in  the  context  of  hardware  verification  [91],  As  we  will  see 
in  Section  3.2.1,  each  iteration  of  SPACER  checks  safety  of  a  different  program  and 
hence,  our  proofs  correspond  to  program  invariants. 

In  particular,  for  a  bound  b  >  0  on  the  length  of  an  execution  (i.e. ,  number  of 
transitions)  and  a  fresh  program  variable  c  denoting  a  down-counter  for  the  number 
of  transitions,  a  formula  tt(v,  c )  is  a  proof  of  bounded  safety  of  P  for  b  iff  the  following 
are  valid: 


l(v )  =>-  7 t(v,c) 

tt(v,  c)  AO  <  c  <  b  A  t(v,  v')  A  c'  =  c  —  1  ==>-  7r(V,  d) 

7r(u,  c)  A  c  =  0  A  errifv )  =>-  _!_ 

In  words,  n  holds  initially  and  for  all  states  reachable  in  at  most  6- many  transitions 
such  that  it  proves  safety.  Intuitively,  7r  is  a  bounded  invariant  for  the  bound  b. 

Given  b  and  n,  one  possibility  for  PBA  is  to  obtain  a  new  program  P  =  ( v ,  f(u), 
t(v,v'),  err{v))  such  that 

1.  l  ==>•  l,  r  ==>-  t,  and  err  err  are  valid,  and 

2.  7T  proves  bounded  safety  of  P  for  b. 
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In  words,  the  initial  condition,  the  transition  relation  and  the  error  condition  are 
weakened  such  that  P  -<  P  and  moreover,  the  proof  7 r  of  bounded  safety  is  preserved. 

However,  to  obtain  more  precise  abstractions,  we  perform  PBA  relative  to  known 
invariants  of  P.  An  invariant  is  a  formula  inv  such  that  the  following  are  valid: 

t(v)  =>■  inviv )  (3.1) 

inv(v)  A  t(v,  v')  =>■  inviv').  (3.2) 

In  other  words,  inv  holds  of  every  reachable  state  of  P.  Note  that  an  invariant  need 
not  be  safe,  i.e. ,  inv{v)  A  err(n)  may  be  satishable. 

Given  an  invariant  inviv )  of  P,  the  goal  of  PBA  relative  to  inv  is  to  obtain 
P  =  (v,  l(v)  A  inv(v),  t(v,v')  A  inviv)  A  inviv'),  err(v)  A  inv(v)),  where,  as  before, 

1.  i  =>■  l,  t  =>■  f ,  and  err  =>■  err  are  valid,  and 

2.  7 r  proves  bounded  safety  of  P  for  b. 

In  words,  we  combine  the  weakening  of  t,  r,  and  err  with  the  invariants  on  the  current 
and  the  next-state  variables.  One  can  easily  show  that  P  P  P.  Using  invariants  of 
P  in  PBA  yields  a  more  precise  abstraction  because  the  reachable  states  of  the 
abstraction  are  confined  to  the  known  invariants. 

3.2.1  Example 

Consider  the  example  transition  system  in  Fig.  3.3.  As  described  above,  the  variables 
b  and  c  in  the  figure  denote  the  bound  on  the  length  of  an  execution  and  a  fresh 
variable  denoting  a  down-counter  for  the  number  of  transitions  along  an  execution, 
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V  =  (x,y,z,w) 

l(v)  =  (x  =  y  =  z  =  w  =  0) 
t(v,  v')  =  \{x'  =  x  +  1  A  y'  =  y  +  100)  V 

(x>4  A  x'  =  x  +  1  A  y'  =  y  +  1)  V 
(y  >  10w  A  z  >  100a;  A 
y'  =  —y  A  x'  =  x)\  A 
w'  =  w  +  1  A  z'  =  z  +  10  A 
0  <  c  <  b  A  d  =  c  —  1 
err(v)  =  c  =  0Aa:>4Ay<2 

Figure  3.3:  An  example  program  P  represented  as  a  transition  system. 

respectively.  Fixing  a  value  of  b  results  in  an  under- approximation  of  the  program. 
For  example,  adding  the  constraint  b  =  0  to  r  in  the  figure  corresponds  to  the 
under-approximation  that  allows  no  transitions.  On  the  other  hand,  adding  the 
constraint  6=1  instead  corresponds  to  the  under-approximation  that  allows  at 
most  one  transition.  While  in  this  example  the  variables  b  and  c  are  part  of  P,  we 
synthesize  such  variables  automatically  in  practice  (see  Section  3.5).  In  the  following, 
we  illustrate  Spacer  using  this  example. 

Bound  and  Solve.  For  6  =  2,  one  possible  proof  of  bounded  safety,  say  7 r2,  is 
obtained  as  the  conjunction  of  the  set  of  clauses  shown  in  Fig.  3.5(a). 

Extract  Invariants.  The  next  step  of  Spacer  is  to  check  if  772  also  proves  un¬ 
bounded  safety  of  P.  For  this  purpose,  we  first  obtain  a  maximal  subset  X-2  of  the 
set  of  clauses  of  112  that  are  invariant  for  P,  i.e. ,  inv  =  A  ^2  makes  the  two  formulas 
(3.1)  and  (3.2)  valid.  We  call  such  a  subset  Z2  a  Maximal  Inductive  Subset  (MIS) 
of  tt2-  Fig.  3.5(b)  shows  one  such  subset  Z2.  In  this  case,  does  not  prove  safety, 
i.e.,  f\X 2  is  satisfiable  with  err.  Nevertheless,  as  a  by-product  of  this  step,  we  have 
obtained  non-trivial  invariants  Z2  of  P. 
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v  =  ( x,y,z,w ) 

Li  (v)  =  (x  =  y  =  z  =  w  =  0)  A 

inv(x,  y ,  z,  w) 

ri  (v,  v')  =  [( xf  =  x  +  1)  V 


v 


v  =  (x,y,z,  w) 

L2  (v)  =  (x  =  y  =  z  =  w  =  0)A 

inv(x ,  y,  z,  ty) 

=  [(x'  =  x  +  1  A  y' =  y  +  100)  V 


‘2(u) 


V 


(x  >  4  A  a/  =  a;  +  1)  V 
(y  >  lOit;  A  z  >  100a;)]  A 
0<c<6  A  c'  =  c  —  1  A 
inv(x,  y ,  z,  tu,  c)  A  inv(x' ,  y' ,  z’ ,  cr) 


(a;  >4  A  a;'  =  a;  +  l  A  ?/  =  y  +  l)V 
(y  >  10t^  A  z  >  100a;)]  A 
0<c<6  A  d  =  c  —  1  A 
inv(x ,  y,  2:,  w,  c)  A  inv(x' ,  y' ,  z' ,  t*/,  c;) 


efri(^)  =  c  =  0  A  a;  >  4  A 
inv(x,  y,  2,  tu,  c) 


err2(v)  =  c  =  0  A  a;>4  A  y  <  2  A 
inv(x,  y,  z,  ay,  c) 


(b)  P2 


Figure  3.4:  Abstractions  Pi  and  P2  of  P  in  Fig.  3.3.  ww  denotes  A  A  for  X2  in  Fig.  3.5(b). 
PBA.  The  next  step  of  Spacer  is  to  perform  PBA  relative  to  known  invariants  Z2. 


Pi  in  Fig.  3.4(a)  is  one  such  abstraction  where  7 r2  continues  to  be  a  proof  of  bounded 
safety  of  P2  for  b  =  2.  In  practice,  we  obtain  the  abstractions  using  an  unsatisfiability 


core  of  the  SMT  problem  used  to  validate  the  proof  of  bounded  safety.  Note  that 
t 1  no  longer  has  the  constraints  on  the  next-state  values  of  z,  y,  and  w  present  in  r 
as  they  are  captured  by  the  invariant  obtained  in  the  previous  step.  In  other  words, 
while  Ti  is  obtained  using  a  structural  (or  syntactic )  abstraction  [16],  the  use  of 
invariants  makes  it  a  more  expressive,  semantic ,  abstraction  mechanism. 

Bound  and  Solve.  SPACER  now  modifies  the  bound  constraint  to  b  =  4  which 
results  in  a  counterexample  using  P\.  One  possible  counterexample  execution  ^4  is 
shown  below  which  corresponds  to  incrementing  x  from  0  to  4: 


(3.3) 


(3, 0, 0, 0, 1, 4),  (4,  3, 0, 0, 0, 4),  (4,  3, 0, 0, 0, 4)) 


where  each  tuple  denotes  a  valuation  of  the  state  variables  (x,y,  z,w,c,b). 

Feasibility  Check  and  Refinement.  ^4  can  be  shown  to  be  infeasible  in  P  and 
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{(z  <  100a;  —  90  V  y  <  lOw) 
2  <  lOOcc,  y  >  0, 
(i<0Vi/>  100)} 


{( z  <  lOOcc  —  90  V  y  <  lOw), 

2  <  100a;,  x  <  2, 

(a;  <  0  V  c  <  1),  {( z  <  100a;  —  90  V  y  <  lOiu), 

(x  <  1  V  c  <  0)}  2  <  100a;} 

(a)  7 r2  (b)  X2  (c)  7t4 

Figure  3.5:  Proofs  and  invariants  found  by  Spacer  for  the  program  P  in  Fig.  3.3. 

Spacer  refines  the  current  abstraction  P\  to  P2,  shown  in  Fig.  3.4(b),  using  CEGAR. 
Bound  and  Solve.  Spacer  checks  bounded  safety  for  b  =  4  with  the  new  abstrac¬ 
tion  P2  and  Fig.  3.5(c)  shows  one  possible  proof  7 r4  as  a  set  of  clauses. 
Proof-of-Safety  Check.  One  can  show  that  7t4  is  MIS  of  itself,  i.e.,  /\  7t4  is  also  an 
invariant  of  P  and  hence,  a  safety  proof  of  P.  At  this  point,  Spacer  terminates  and 
outputs  SAFE. 

While  we  have  carefully  chosen  the  values  of  the  bound  to  save  space,  the  abstrac¬ 
tions,  proofs,  and  invariants  shown  above  were  all  computed  automatically  by  our 
implementation  starting  with  the  initial  bound  6  =  0  and  incrementing  the  bound 
by  1,  each  iteration.  Even  on  this  small  example,  abstraction  improves  the  runtime 
by  five  times. 

3.3  Preliminaries 

The  transition  systems  we  have  seen  in  Section  3.2  are  simplistic  and  hide  the  control 
structure  of  the  input  program.  In  this  section,  we  consider  a  more  general  graph 
representation  of  a  program  that  explicates  the  control  structure.  As  mentioned 
earlier,  we  restrict  ourselves  to  while-programs.  As  in  Section  2.3,  consider  a  first- 
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v  =  { x,y,z,w ) 

i(v)  =  (x  =  y  =  z  =  w  =  0) 
t(v,v')  =  \(x'  =  x  +  1  A  y'  =  y  4-  100)  V 

(x  >4  A  x'  =  x  +1  A  y'  =  y  +  1)  V 
(y  >  lOio  A  z  >  100a;  A 
y'  =  — y  /\  x'  =  a;)]  A 
u>'  =  iu  +  l  A  z'  =  z  +  10  A 
0  <  c  <  6  A  d  =  c  —  1 
err(y )  =  c  =  0Aa;>4Ay<2 

Figure  3.6:  A  graph  representation  of  the  transition  system  in  Fig.  3.3. 

order  language  with  signature  S  and  let  Th  be  an  5-theory. 

Definition  3  (Program).  A  program  P  is  a  tuple  (L,  £°,  £e,  V,  r)  where 

1.  L  is  a  finite  set  denoting  the  control  locations, 

2.  1°  G  L  and  le  G  L  are  the  unique  initial  and  error  locations, 

3.  V  is  a  finite  set  of  program  variables  disjoint  from  S,  and 

4-  t  is  a  map  from  pairs  of  locations  in  Lx  L  to  quantifier-free  sentences  over  the 
signature  (5  U  V  U  V'),  where  V'  is  obtained  from  V  by  priming  each  variable 
in  V  and  denotes  the  next- state  values  of  the  variables  in  V. 

Intuitively,  r(£i,£j)  is  the  relation  between  the  current  values  of  V  at  £{  and  the 
next  values  of  V  at  £j  on  a  transition  from  £i  to  £j.  We  refer  to  r  as  the  transition 
relation.  Without  loss  of  generality,  we  assume  that  there  is  no  incoming  transition 
to  the  initial  location  £°  and  no  outgoing  transition  from  the  error  location  £e,  i.e. , 
for  all  £  G  L,  t(£,  £°)  =  _L  and  r(£e,  £)  =  _L.  We  refer  to  the  components  of  a  program 
P  by  a  subscript,  e.g.,  Lp  denotes  the  set  of  locations  of  P. 
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For  example,  Fig.  3.6  shows  a  program  (L,  £°,  £e,  V,  r)  corresponding  to  the  transi¬ 
tion  system  in  Fig.  3.3,  where  L  =  {en,  Ip,  er},  £°  =  en,  £e  =  er,  V  =  {x,  y,  z,  w ,  c,  b}, 
r(en ,  Ip)  =  I,  r[lp,  Ip)  =  T,  r(lp,  er)  =  E. 

Let  P  —(L,  £°,  £e ,  V,  r )  be  a  program.  A  control  path  of  P  is  a  finite6  sequence 
of  control  locations  (£°  =  £$,  l\, . . . ,  £k),  beginning  with  the  initial  location  £°,  such 
that  T(£i,£i+i)  7^  _L  for  0  <  i  <  k.  A  state  of  P  is  an  assignment  to  the  variables  in 
V.  A  control  path  (£°  =  £0,  £1,...,  £k)  is  called  feasible  iff  there  is  an  5-structure  I 
with  I  \=  Th  and  a  sequence  of  states  (s0,  Si, . . . ,  sk)  such  that 

I{V  h- >  Si}{V'  |=  r(£i,£i+i),  for  all  0  <  i  <  k  (3.4) 

i.e. ,  the  sequence  of  states  is  an  execution  along  the  control  path  (the  reader  is 
referred  to  Section  2.3  for  the  notation  used  above). 

For  example,  (en,  Ip,  Ip,  Ip)  is  a  feasible  control  path  of  the  program  in  Fig.  3.6  as, 
under  the  standard  interpretation  of  the  arithmetic  symbols,  the  sequence  of  states 
((0,0,  0,0,  0,0),  (0,0,  0,0,  2,  2),  (1,100,1,10,  1,2),  (2,200,2,20,0,2))  satisfies  (3.4). 

A  location  £  is  reachable  iff  there  exists  a  feasible  control  path  ending  with  £. 
P  is  safe  iff  £e  is  not  reachable.  For  example,  the  program  in  Fig.  3.6  is  safe,  as 
shown  in  the  previous  section.  P  is  said  to  be  decidable  iff  the  safety  problem  of  P 
is  decidable.  For  example,  the  program  U  obtained  from  P  in  Fig.  3.6  by  replacing 
b  with  5  is  decidable  because  (a)  U  has  finitely  many  feasible  control  paths,  each  of 
finite  length,  and  (b)  the  formulas  only  use  linear  arithmetic  (integers  or  rationals) 
which  is  decidable. 

6This  suffices  as  we  only  deal  with  safety  properties. 
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We  will  now  define  invariance  and  proof  of  safety. 

Definition  4  (Invariant  Map).  An  invariant  map  for  P  is  a  map  I  from  locations 
to  sets  of  sentences  over  the  signature  5  U  V  such  that  the  following  hold: 

1-  |=  Th  T  =>•  /\n{l°),  and 

2.  for  every  £h£j  e  L,  \=Th  {/\n (£f)  A  r(£h£j))  =>•  A  ^(A)'- 

One  can  show,  using  a  least  fixed-point  characterization,  that  given  an  invariant 
map  X,  T{£)  over-approximates  the  reachable  states  at  a  location  l. 

Definition  5  (Safety  Proof).  A  safety  proof  for  P  is  an  invariant  map  tt  that  is  also 
safe,  i.e.,  \=Th /\n(£e)  =>•  X. 

For  example,  Fig.  3.5(c)  shows  a  safety  proof  for  the  program  in  Fig.  3.6,  as  also 
discussed  in  the  previous  section.  As  the  formulas  given  by  an  invariant  map  over¬ 
approximate  the  reachable  states,  safety  of  the  map  implies  safety  of  the  program. 

A  counterexample  to  safety  is  a  triple  (£,  I,  s)  where  £  is  a  feasible  control  path  in 
P  ending  with  £e,  I  is  an  5-structure  with  I  \=  Th ,  and  s  is  a  sequence  of  states  that 
satisfy  (3.4).  For  example,  ((en,  Ip,  Ip,  Ip,  Ip,  Ip,  er),  I,  X4),  where  /  is  the  5-structure 
that  interprets  the  arithmetic  symbols  in  the  standard  way  and  is  as  shown  in 
(3.3)  is  a  counterexample  to  safety  for  the  program  P\  in  Fig.  3.4(b). 

Definition  6  (Abstraction  Relation).  Given  two  programs,  Pi  =  (Li,  £°,  £\,  V\,  rf) 
and  P2  =  {L2,  £2,  £\i  V2,  T2) j  P2  is  an  abstraction  {i.e.,  an  over- approximation)  of 
Pi  via  a  total  function  a  :  Li  — >  L2,  denoted  Pi  -<a  P2,  iff 

1.  Vi  =  V2, 

2.  cr(£°)  =  £2  and  a{£\)  =  £\,  and 

3.  for  every  £i,£j  e  Li,  \=Th  n(£i,£j)  =>■  T2(cr(£i),a(£j)). 
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In  this  case,  we  also  say  that  Pi  is  an  under-approximation  ( i.e a  refinement)  of 
P2  and  call  a  an  abstraction  function.  We  say  that  P2  strictly  abstracts  Pi  via  cr, 
denoted  Pi  -<a  P2,  iff  Pi  -<a  P2  and  there  is  no  function  v  :  L2  — >  Li  such  that 
P2  -<v  Pi.  When  a  is  clear  from  the  context  or  unnecessary,  we  drop  the  subscript. 

That  is,  P2  abstracts  Pi  iff  there  is  a  total  map  cr  from  Li  to  L2  such  that 
every  feasible  transition  of  Pi  corresponds  (via  cr)  to  a  feasible  transition  of  P2.  For 
example,  if  Pi  is  a  finite  unrolling  of  P2,  then  a  maps  the  locations  of  Pi  to  the 
corresponding  ones  in  P2.  For  an  example  of  strict  abstraction,  it  can  be  shown  that 
P  -<i(i  Pi,  where  P  is  in  Fig.  3.6,  P\  is  in  Fig.  3.4(a),  and  id  denotes  the  identity 
relation. 

One  can  easily  show  that  the  above  notion  of  abstraction  is  proof-preserving,  i.e., 
if  Pi  ■<  P2,  then  a  safety  proof  7 r  of  P2  is  also  a  safety  proof  of  Pi. 

For  two  transition  relations  T\  and  t2  over  a  set  of  locations  L ,  we  write  T\  ==>•  t2 
to  denote  that  for  every  ix,  I2  6  L,  \=Th  Ti(£i,£2)  =>-  t2(£ i,£2).  We  also  write  Ti  At2 
to  denote  the  point-wise  conjunction  of  the  two  transition  relations. 

We  extend  a  :  Li  — >  L2  from  locations  to  control  paths  in  the  straightforward 
manner.  For  a  counterexample  ^  =  ( I,I,s ),  we  define  cr(fif)  =  ( a(£),I,s ).  For  a 
transition  relation  r  on  L2l  we  write  a(r)  to  denote  an  embedding  of  r  into  Li  via 
cr,  defined  as  follows:  for  locations  £i,£2  G  Li,  cr(r)(£ i,£2)  =  r(cr(i?1),  a(£2)).  For 
example,  in  the  definition  above,  if  Pi  -<a  P2,  then  Ti  ==>  cr(r2). 

In  the  following,  to  avoid  clutter,  we  assume  a  fixed  5-theory  Th  and  we  write 
|=  to  mean  \=Th-  Also,  every  (5  U  X)-structure  we  consider,  for  an  arbitrary  set  of 
new  symbols  X ,  is  assumed  to  be  a  model  of  Th. 
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3.4  The  Algorithm 


In  this  section,  we  describe  our  algorithm  SPACER  at  a  high-level,  where  the  de¬ 
scription  of  several  routines  is  confined  only  to  their  interfaces.  Our  specific  imple¬ 
mentation  choices  are  described  in  Section  3.5.  Figs.  3.7  and  3.8  show  the  pseudo¬ 
code  of  the  algorithm.  The  routine  SPACER  checks  safety  of  a  given  program  P  = 
(Lp,  t°p,  £p,  Vp,  Tp).  The  algorithm  maintains  (a)  an  invariant  map  I  (see  Defini¬ 
tion  4),  (b)  an  abstraction  A  of  P,  (c)  a  decidable  under-approximation  U  of  A,  and 
(d)  a  function  a  such  that  U  -<a  A.  The  abstraction  A  has  the  same  set  of  control 
locations  as  P  and  satisfies  P  A,  i.e. ,  A  differs  from  P  only  in  its  transition  re¬ 
lation.  We  write  Ap  to  denote  the  restriction  of  A  to  the  invariant  map  X,  obtained 
by  strengthening  ta  to  A^i,  £2  •  X(fA)  A  ta{£ i,  £2)  A  X(f'2)/ .  Similarly,  we  write  Up, a  to 
denote  the  restriction  of  U  to  the  invariant  map  X  of  A,  obtained  by  strengthening 
t p  to  A£i ,£2  •  X(cr(£i))  A  £2)  A  l{a(i2)y.  When  A  is  clear,  we  simply  write 

Up.  Spacer  assumes  the  existence  of  an  oracle,  Solve,  that  decides  whether  Up  is 
safe  and  returns  either  a  safety  proof  or  a  counterexample  (see  Section  3.5  for  an 
implementation  of  Solve). 

Spacer  initializes  the  abstraction  A  of  P  and  an  under-approximation  U  of 
A,  using  InitAbs  and  InitUnder,  respectively  (lines  1-2).  It  then  initializes  the 
invariant  map  X  to  the  empty  map  (line  3).  Each  iteration  of  the  main  loop  (at 
line  4)  checks  whether  Up  is  safe,  for  the  current  values  of  U  and  X,  using  Solve 
(line  5).  If  Up  is  safe  with  a  proof  n,  it  is  then  checked  whether  a  safety  proof  of 
the  original  program  P  can  be  obtained  using  n,  as  follows.  First,  n  is  mined  for 
new  invariants  of  P  using  EXTRACTlNVS  (line  7).  Then,  if  the  invariants  at  the 
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error  location  £ep  are  unsat isfiable,  it  means  that  the  error  location  is  shown  to  be 
unreachable  and  Spacer  returns  Safe  (lines  8-9).  Otherwise,  the  abstraction  A  is 
updated  to  a  new  Proof  Based  Abstraction  via  Pba,  and  a  new  under-approximation 
is  constructed  using  NextUnder  (lines  10-11).  If,  on  the  other  hand,  Ux  is  unsafe 
at  line  4,  the  obtained  counterexample  ^  is  validated  using  Cegar  (line  13).  If  c& 
is  feasible  in  P,  SPACER  returns  UNSAFE  (line  15);  otherwise,  both  A  and  U  are 
refined  (see  the  description  of  CEGAR  below). 

In  the  following,  we  give  a  brief  description  of  the  routines.  Let  U  =  (Lp,  £p,  £p,  Vu, 
Tjj) ,  U  Aa  A,  and  A  =  (La,  £%  £%  Va,  Ta).  Note  that  La  =  Lp  as  mentioned  above. 
EXTRACTlNVS  has  two  high  level  steps:  (a)  use  the  proof  tt  of  U  to  obtain  a  con¬ 
junction  of  formulas  at  each  location  of  P,  and  (b)  compute  the  maximal  subset 
of  those  conjuncts  at  each  location  that  are  together  invariant  (according  to  Def¬ 
inition  4).  To  obtain  the  conjunctions  at  a  given  location  in  Lp,  we  collect  the 
formulas  given  by  the  proof  tt  of  U  at  all  corresponding  locations  (w.r.t.  cr)  in  Lp 
(lines  17-18),  take  their  disjunction,  and  convert  to  conjunctive  form  (not  necessarily 
conjunctive  normal  form;  see  lines  19-20).  The  intuition  behind  taking  a  disjunction 
is  that  the  various  locations  £u  G  Lp  with  a(£u)  =  £  G  Lp  represent  different  subsets 
(not  necessarily  exhaustive)  of  the  reachable  states  at  i.  In  our  implementation,  we 
only  have  one  such  corresponding  location  (see  Section  3.5).  To  obtain  the  maximal 
subsets  that  are  invariant,  we  use  a  straightforward  greatest  fixed-point  computation 
(lines  21-22),  similar  to  the  Houdini  approach  [54], 
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g}ohal(program  P) 
global(  invariant  map  I  of  P) 


S  PACER  () 

1  A  4-  InitAbs(P) 

2  (U,a)  4-  InitUnder(A) 

3  X  empty  map 

4  while  true  do 

5  (result,  7r,  If)  4—  Solve([/x) 

6  if  result  is  Safe  then 

r  I<-lU  ExtractInvs(A,  U,  a,  i r) 

is  unsatisfiable  then 
9  return  Safe 


10 

ii 


(A,U)  4-  Pba(A,  U,  a,  7r) 

(C7,  cr)  <-  NextUnder(A,  U,  a) 


12 

13 

14 

15 


else 

(feas,  A,  U)  4-  Cegar(A,  U,  a,  f ) 
if  feas  then 
[  return  Unsafe 


16 

17 

18 


ExtractInvs(A,  U,  a,  n) 

1Z,  C  4—  empty  maps  from  locations  in  Lp  to  sets  of  sentences  over  <S  U  Vp 
for  £  £  Ljj  do 
|  a,dd  /\tt(£)  to  C(a(£)) 


19 

20 

21 

22 


for  £  £  Lp  do 

j  1Z(£)  4—  conjuncts(\/  C(£)) 

while  exist  £i,£j  £  Lp,tp  £  1Z(£j)  s.t.  (7Z(£i)  /\X(£f)  Arp(£i,£j)  =>■  y(Vp))  do 

L  n(ei)  ■■=  n^o)  \  M 


23 


return  1Z 


24 


NextUnder(A,  U,  a) 

return  U  s.t.  U  -<ai  U  A,  a  =  <72  °  <Ji  and 
Strengthen^,  <t(tp))  -<;  Strengthen(U,  a2(rP)) 


Figure  3.7:  Pseudo-code  of  Spacer,  except  the  routines  for  Pba  and  Cegar  which  are  given  in 
Fig.  3.8. 
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Pba  updates  the  abstraction  A  of  P  by  using  the  safety  proof  n  of  U  as  follows. 
As  U  -<a  A,  Tu  =>  ct(ta)  holds  and  assume  without  loss  of  generality  that  Tp 
is  of  the  form  cr(r^)  A  p,  for  some  p  (i.e.,  one  can  always  equivalently  rewrite  Tp  to 
this  form).  Let  W  be  obtained  from  U  by  strengthening  the  transition  relation  with 
cr(rp)  (using  STRENGTHEN  on  line  26).  Clearly,  Tw  =  cr (tp)  A  p.  It  is  easy  to  see 
that  7 r  is  also  a  proof  of  W.  An  abstraction  W  of  W  is  then  obtained  such  that 
(a)  =  cr(fp )  A  p  where  fp  and  p  respectively  abstract  Tp  and  p,  (b)  for  the  new 

abstraction  A  obtained  by  replacing  the  transition  relation  of  A  with  fp,  n  is  a  safety 
proof  of  Wj  &  (line  27).  That  is,  A  is  obtained  as  a  proof-based  abstraction  of  P 
using  the  proof  7T  of  U  and  the  currently  known  invariants  I. 

NextUnder  returns  the  next  under- approximation  U  of  A  to  be  checked  for  safety. 
We  require  that  the  abstraction  functions  between  U,  U,  and  A  compose  so  that  the 
corresponding  transitions  in  U  and  U  map  to  the  same  transition  of  the  common 
abstraction  A.  To  ensure  progress,  we  require  U  ~<U .  Moreover,  to  ensure  progress 
in  checking  safety  of  P,  we  also  require  the  last  condition  on  line  24.  Intuitively,  we 
require  U  to  also  have  more  concrete  behaviors  than  U .  If  this  were  not  possible, 
safety  of  U  would  have  implied  safety  of  P  and  Spacer  would  have  terminated. 

CEGAR  checks  if  the  counterexample  ^  exhibits  a  feasible  behavior  in  P,  using 
IsFeasible  (line  29).  If  CA  is  feasible,  Cegar  returns  saying  so  (line  34).  Otherwise, 
%J  is  spurious  and  the  abstraction  A  is  refined  to  A  by  eliminating  (and  possibly 
more  spurious  behaviors)  (line  31).  This  is  obtained  by  strengthening  the  transition 
relation,  i.e.,  A  -<l(i  A  holds.  Finally,  the  under-approximation  U  is  strengthened 
with  the  refined  transition  relation  of  A  (using  STRENGTHEN  on  line  32),  such  that 
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Pba(A,  U,  a,  7 r) 

25  let  p  be  s.t.  tjj  =  cf(ta)  A  p 

26  W  <-  Strengthen([/,  cr(Tp)) 

27  choose  W  s.t.  W  <W  and  =  cr(fp)  A  p  with  rp  =>  fp,  p  =>  p,  tt  is  a  safety 
proof  of  Wy-  where  A  =  A[ta  <—  fp] 

28  return  (A,  VF) 

Cegar(A,  t/,  <7, 

29  /eas  IsFeasible(ct(<^),P) 

30  if  not  feas  then 

31  let  A  -<id  A  s.t.  -iIsFeasible(ct(<^),  Ax) 

32  U  4-  Strengthen (U,  <j{ta)) 

33  return  (false,  A,  U) 

34  return  (true.  None,  None) 

Figure  3.8:  Routines  for  Proof  based  Abstraction  and  CEGAR. 

the  resulting  U  satisfies  U  A  for  the  same  abstraction  function  a. 

In  the  following,  we  show  the  soundness  and  progress  guarantees  of  Spacer. 
Lemma  7  (Invariant  Maps).  X  is  always  an  invariant  map  for  P. 

Proof.  Initially,  I  is  empty  and  X(£)  =  T  for  every  location  i  G  LP.  Clearly,  X  is  an 
invariant  map.  When  EXTRACTlNVS  returns  on  line  23,  the  guard  of  the  while  loop 
at  line  21  fails  to  hold  which  implies  that  the  map  1Z  is  also  an  invariant  map.  It 
follows  that  when  X  is  updated  on  line  7,  it  remains  an  invariant  map.  □ 

Soundness  is  now  immediate: 

Theorem  8  (Soundness).  P  is  safe  ( unsafe )  if  Spacer  returns  Safe  (Unsafe). 

To  show  progress,  we  start  with  a  useful  lemma.  In  the  following,  we  some¬ 
times  refer  to  the  components  of  a  program  P  by  application,  in  addition  to  using 
subscripts,  e.g.,  L(P)  denotes  the  locations  of  P. 
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Lemma  8.  Let  U\  -<ai  A  and  let  p\  be  such  that  r{U\)  =  O]  (ta)  A  p\.  If  U\ 

U 2  Aa2  A  with  o\  =  o2  °  p,  then  there  exists  p2  such  that  r{U2)  =  cr2(rA)  A  p2  and 
Pi  =>  h(.P2)- 

Proof.  As  U2  -<a2  A,  we  know  that  r(U2)  =>■  o2(ta )  (Definition  6).  So,  there  exists 
p  such  that  t(U2 )  =  ct2(ta)  Ap.  As  U\  U2,  we  also  know  that  r(f/i)  p{r{U2)). 
Together  with  <j\  =  o2  o  p,  we  obtain 

ffiW  Api  cti(ta)  a  p(p).  (3.5) 


Consider 

pi  =  pv\ile*euu2).l  V  /;l(<,1)  =  ^AM(’31)  =  <?Ap1(c,1,c‘) 

Intuitively,  p2  captures  all  the  transitions  that  must  be  feasible  in  U2  as  guaranteed 
by  the  relationship  U\  -<n  U2. 

It  can  be  easily  shown  that  p\  =>-  p(p2). 

It  remains  to  show  that  o2(ta)  A  p2  is  equivalent  to  r{U2)  =  <t2(ta)  A  p.  That  is, 
we  need  to  show  that,  for  all  £‘f ,  P-  G  L([/2), 


|=  (a2(rA)  A  p)  (*?,  £j)  <=►  (<r2(rA)  A  p2)  (£ 


The  left  to  right  direction  is  obvious  as  |=  p  ==>-  p2  from  the  definition  of  p2. 

For  the  other  direction,  assume  for  the  sake  of  contradiction  that  there  exist 
G  L(U2 )  and  an  S  U  V  U  D'-structure  I  (with  I  \=  Th )  such  that  /  |= 
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Ui  dlai  Ai 

(pi) 


PBA(Ai,Ui,7T ) 


Ui  ~^p  Ui+ 1  z^o-j+i  ^4-i+i 
(pO  (pi+i) 

NextUnder 
(Ai+ 1,  C/j) 

(a)  Ui  is  safe. 


Ut  ^  Ai 

(Pi) 


Cegar  (A^Ui^i) 


(b)  Ui  is  unsafe. 


Figure  3.9:  Relation  between  two  successive  under-approximations  Ui  and  Ui+ 


(^(r)  A  p2)  (^,^)  and  I  p(£f,£2).  From  the  definition  of  p2,  it  follows  that 
there  exist  £},£]  6  L(U\)  such  that  /  |=  pi(£\,£}j),  and  p{£\)  =  if  and  p(£j)  = 
£2.  Moreover,  we  know  that  I  (=  Ta(ct2(^?),  cr2(^))-  As,  ay  =  cr2  o  /i,  it  follows 
that  /  |=  7_A(cr1(£f),  cri(£f)).  So,  we  have  found  €  L(U\)  such  that  /  |= 

(cti(ta)  A  pi)  (£•,£]).  From  (3.5),  it  follows  that  /  |=  p(p)(£j,  £]),  i.e.,  I  (=  p(£f,£2) 
which  contradicts  our  assumption.  □ 


Theorem  9  (Progress).  Let  Ai,  Ui,  and  c€x  be  the  values  of  A,  U,  and  in  the  ith 
iteration  of  Spacer  with  Ui  Aa.  Ai  and  let  Ui  denote  the  concretization  of  Ui,  i.e., 
Ui  —  Strengthen^,  cr^Tp)).  Then,  if  Ui+i  exists, 

1.  if  Ui  is  safe,  then  Ul+\  has  strictly  more  concrete  behaviors,  i.e.,  Ui  -<  Ul+  \ , 

2.  if  Ui  is  unsafe,  C/j+i  has  the  same  concrete  behaviors,  i.e.,  Ui  Aid  U+\  and 
Ui-\- 1  A id  Ui,  and 

3.  ifUi  is  unsafe,  %  does  not  repeat  in  future,  i.e.,  for  every  j  >  i,  (Jjif&j)  ^  a i(f&i). 

Proof.  1.  Ui. |_i  is  obtained  from  Ui  after  a  call  to  Pba  followed  by  NextUnder, 
as  shown  in  Fig.  3.9(a).  For  Uj ,  the  figure  also  shows  pj  in  brackets  such  that 
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r{Uj)  =  aj(r(Aj))  A  pj  (this  is  always  possible  as  t(Uj)  =>-  aj(r(Aj))). 

Pba  ensures  that  pi  =>•  pi  and  Lemma  8  guarantees  the  existence  of  a  pi+ 1 
with  pi  /i(pi+1).  Together,  we  have  pt  ==>-  p(pi+i).  Furthermore, 

NextUnder  requires  a*  =  (Tj+i  o  p.  Then,  f/*  ^  Ul+\ ,  as  shown  below. 


r(f/i)  =  cri(rp)  A  pj 


(o-i+i  op)(rP)  A  p(pi+i) 
p((Ti+i(rP))  A/i(pi+i) 
p(r(f7i+i)) 


To  show  that  Ui  -<  f/j+i,  assume  for  the  sake  of  contradiction  that  f/j+1  Ui. 
Then,  as  pi  =>■  /5i, 


r(f/i+i)  =  ai+i(rP)  A  pi+i 

=►  w(ai(rp)  A  pi) 

=k  cu((Ti(rp)  A  pi) 

=  o;(r(f/i)) 

giving  us  Ul+i  Up  This  contradicts  Ut  -<  Ui+\  on  line  24  of  Fig.  3.7. 

2.  Ui+ 1  is  obtained  from  Ui  after  a  call  to  Cegar  as  shown  in  Fig.  3.9(b).  Again, 
for  Uj,  the  figure  shows  pj  in  brackets  such  that  r(Uj)  =  aj(r(Aj))  Apj.  CEGAR 


ensures  that  p*  =  pi+ i  and  cq  =  cq+1.  These  imply  that  r(f/j)  <£=>•  r(f/i+1). 

3.  Let  ^  =  (£i,  /,  s).  We  prove  the  stronger  statement  that  for  every  j  >  i,  there 
exist  a  control  path  lj  in  Uj  and  a  pj  such  that  t(U3)  =  aj(r(Aj))  A  pj,  and 
the  following  hold: 

(a)  aj(£j)  =  (Tifc), 

(b)  ( £j,I,s )  is  a  counterexample  for  Uj  when  the  transition  relation  is  re¬ 
stricted  to  pj  but  not  when  restricted  to  a3  (r  (A3 ) ) . 

In  words,  we  show  that  the  control  path  of  %,  corresponds  to  a  control  path  in 
every  future  Uj  (via  <jj)  and  the  state  sequence  s  is  feasible  when  restricted  to 
Pj  but  not  when  restricted  to  a3(r(Aj)).  The  latter  is  sufficient  to  show  that 
Uj  does  not  admit 

We  prove  this  by  induction  on  j.  If  j  =  i  +  1,  Fig.  3.9(b)  shows  the  relation 
between  Ut  and  Ut+i.  Again,  Cegar,  ensures  that  pi  =  pi+\  and  cq  =  al+ 1 . 
The  required  control  path  i3  in  (a)  is  the  same  as  U.  Also,  Cegar  ensures 
that  r(Aj)  does  not  admit  satisfying  (b). 

Now,  assume  that  U3  satisfies  (a)  and  (b),  for  an  arbitrary  j.  We  show  that  U3+\ 
also  satisfies  (a)  and  (b).  If  U3+\  is  obtained  from  Uj  after  a  call  to  Cegar, 
the  argument  is  the  same  as  for  the  base  case  above.  The  other  possibility  is  as 
shown  in  Fig.  3.9(a)  where  Uj  is  safe  and  U3+\  is  obtained  after  a  call  to  Pba, 
followed  by  a  call  to  NextUnder.  Consider  pj,  pj  and  p3+i,  similar  to  pi;  pi 
and  pi+\  in  the  figure.  Note  that  PBA  ensures  that  pj  =>-  pj. 

To  see  that  (a)  is  satisfied,  consider  the  control  path  p(£j)  and  note  that  cxj  = 
crj+ 1  o  p  (line  24  of  Fig.  3.7). 
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To  see  that  (b)  is  satisfied,  Lemma  8  ensures  the  existence  of  pJ+\  with  pj  ==>- 
p(pj+i).  As  s  is  feasible  along  £j  when  the  transition  relation  is  restricted  to 
Pj  and  hence,  for  pj,  it  is  also  feasible  along  p(£j)  when  the  transition  relation 
is  restricted  to  p(pj+ 1).  Moreover,  s  is  infeasible  along  £j  for  the  restriction 
aj(r(Aj+ 1)),  as  Uj  is  safe.  Hence,  it  remains  infeasible  along  p(£j)  for  the 
restriction  aj+\{r{Aj+i ))  (follows  from  aj+ \  o  p  =  aj). 

o 

In  this  section,  we  presented  the  high-level  structure  of  Spacer.  As  we  have  seen 
above,  we  only  presented  an  interface  for  the  routines  InitUnder,  EXTRACTlNVS, 
Pba,  NextUnder,  Cegar,  IsFeasible.  In  the  next  section,  we  complete  the 
picture  by  describing  the  implementation  used  in  our  prototype. 

3.5  Implementation 

Let  P  —( L ,  £°,  £e,  V,  r)  be  the  input  program.  First,  we  transform  P  to  P  by 
creating  new  counter  variables  for  the  loops  of  P  and  adding  extra  constraints  to 
the  transition  relation  in  order  to  count  the  number  of  loop  iterations,  as  described 
below. 

As  the  first  step,  we  construct  a  Weak  Topological  Order  (WTO)  [24]  of  P ,  which 
is  a  well-parenthesized  total  order  of  its  locations  L  without  two  consecutive  open 
brackets,  denoted  <,  satisfying  the  following  condition.  Let  the  locations  within  a 
matching  open-close  bracket  pair  constitute  a  component  and  let  the  smallest  location 
w.r.t  <  in  a  component  be  its  head.  Let  hds(£ )  be  the  outside-in  list  of  the  heads  of 
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TB  (-Bl)=T 
tb(E2)  =  (c'1=c1) 


tb(Xi)  =  (ci—c1  a 

c2=0) 

tb(x 2)  =  (ci=0  A 
c2=0) 


tb(Ri)=(ci=ci  a 

0<C2<&2  A 
c2  =c2  —  1) 

TB  (-R2)  =  (0<ci<6i  A 
ci=ci  — 1) 


Figure  3.10:  Program  with  a  nested  loop  and  its  corresponding  bounded  transition  constraints, 
components  containing  l.  Let  £\  <  £2  be  defined  as  (0  =  4  V  fi  <  £2)-  Then, 


V4  £j  £  L  ■  r(£i,  £j)  A  ^  <  £i  =►  ^  G  /ids(4)  (3.6) 

Intuitively,  <  is  a  total  order  of  L  such  that  each  component  identifies  a  loop 
in  P,  the  head  of  a  component  identifies  the  entry  location  of  the  loop  and  hds(£ ) 
identifies  the  outside-in  list  of  nested  loops  containing  £.  Condition  (3.6)  says  that 
a  back-edge ,  w.r.t  <,  leads  to  the  head  of  a  component  containing  the  source  of  the 
edge,  denoting  the  start  of  a  new  iteration  of  the  corresponding  loop.  For  example, 
Fig.  3.10  shows  a  program  with  two  loops,  an  outer  loop  (4,  £ 2 ,  £3)  and  an  inner 
loop  (£2).  One  possible  WTO  for  this  program  is  £T°(t'i(£2)^3)^e”  with  £\  and  £ 2  as 
the  heads  of  the  two  components. 

Note  that  the  above  definition  of  WTO  is  non- deterministic  and  there  are  multiple 
ways  of  implementing  such  an  ordering.  Without  loss  of  generality,  assume  that  1° 
is  always  the  smallest  and  £e  is  always  the  largest  location  of  a  WTO. 

Bound  Variables.  Next,  we  introduce  a  set  C  of  rational  variables,  one  per  head  of 
a  component,  and  the  corresponding  partial  mapping  ctr  :  L  — 1  C.  Intuitively,  ctr(t) 
is  the  number  of  iterations  (completed  or  remaining,  depending  on  whether  we  are 


91 


counting  up  or  down,  respectively)  of  the  component  whose  head  is  £.  Also,  let  B  be 
another  set  of  rational  variables  and  let  bound  :  C  — »  B  be  a  bijection  (i.e.,  \B\  =  IC'D- 
Informally,  bound(c)  denotes  the  upper  bound  of  c.  For  example,  in  Fig.  3.10  we 
have  C  =  {ci,c2},  C\  =  ctr{£\),  c2  =  ctr{£2),  B  =  {61,62})  bound(ci )  =  6l5  and 
bound(c2)  =  b2.  We  construct  a  bounded  program  P  =  (L,  £°,  le ,  V UCUB,  f),  where 
W  i,  £j  G  L  ■  f(£i,  £j)  =  r(£i,  £j)  A  rs(£i,  £j)  and  tb(£i,  £j)  is  a  set  of  constraints  defined 
as  follows.  Let  c3  =  ctr(£j)  and  bj  =  bound(cj).  We  only  describe  the  constraints  for 
counting  down  the  number  of  iterations  from  an  initial  value,  respecting  the  bounded 
given  by  the  bounding  variables. 

Entry:  £^  <  £j  and  l3  is  a  head,  i.e.,  entering  a  new  component  (e.g.,  E\  and  E2  in 
Fig.  3.10).  Then,  TB(£i,£j )  contains  a  constraint  corresponding  to  Cj  being  assigned 
non-deterministically. 

Re-entry:  £j  <  £%.  i.e.,  re-entering  a  component  via  a  back-edge  (e.g.,  R\  and  R2 
in  Fig.  3.10).  Then,  rB(£i,£j )  contains  the  constraint  (0  <  c'-  A  c'  =  Cj  —  1  A  Cj  <  bj, 
i.e.,  it  decrements  c3  as  long  as  it  is  not  zero. 

Exit:  £i  <  £jAhds(£i)  D  hds(£j),  i.e.,  exiting  (one  or  more)  components  containing 
£t  (e.g.,  Xi  and  A"2  in  Fig.  3.10).  Then,  for  each  h  G  hds(£j)  \  hds(£j),  Ts(£i,£j) 
contains  the  constraint  ctr{h )  =  0. 

Pass-on.  For  each  h  G  hds(£j)  \  {£j},  Tr(A,  £j)  contains  the  constraint  ctr(h)  = 
ctr{h)' .  Thus,  when  the  transition  is  inside  a  component  the  current  value  of  its 
counter  is  remembered.  See  Tb  for  the  transitions  E2,  Ri  and  X\  in  Fig.  3.10. 

In  other  words,  a  counter  is  assigned  a  non-deterministic  initial  value  when  en¬ 
tering  its  component,  and  decremented  until  zero  before  exiting.  Since  the  bound 
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variables  (i.e.,  B)  are  unconstrained,  P  and  P  are  equivalent  w.r.t.  safety,  as  shown 
below. 

Lemma  9.  P  is  safe  iff  P  is  safe. 

Proof.  To  show  that  safety  of  P  implies  safety  of  P,  we  prove  its  contrapositive. 
Assume  P  is  unsafe.  So,  there  exists  a  counterexample  to  safety  {£,  I,s).  As  Tp  =  r  A 
Tb,  it  is  obvious  that  projecting  the  state  sequence  onto  V  gives  us  a  counterexample 
to  safety  for  P.  So,  P  is  unsafe  as  well. 

Now  suppose  that  P  is  unsafe  with  a  counterexample  ^  =  (£,  /,  s).  Then,  we  can 
extend  ^  to  a  counterexample  c€  of  P  as  follows:  (a)  for  each  loop  of  P  and  for  each 
time  the  loop  is  entered  and  exited  along  count  the  number  of  iterations,  say  n, 
(b)  assign  values  to  the  corresponding  counter  variables  along  the  control  path  to 
simulate  a  count-down  from  n  to  0,  and  (c)  for  the  bound  variable  corresponding  to 
the  loop,  assign  a  value  greater  than  or  equal  to  the  maximum  number  of  iterations 
of  the  loop  along  .  So,  ^  is  also  unsafe.  □ 

Now,  given  a  safety  proof  of  P,  one  can  transform  it  to  a  proof  of  P  by  universally 
quantifying  all  the  bound  and  counter  variables.  See  Appendix  3. A  for  a  proof.  We 
can  thus  check  safety  of  P  in  order  to  decide  safety  of  P. 

In  the  rest  of  this  section,  we  describe  our  abstractions  and  under-approximations 
of  P,  followed  by  our  implementation  of  the  various  routines  in  Fig.  3.7. 
Abstractions.  Recall  that  rp  =  t  A  tb-  Let  E  be  a  set  of  fresh  Boolean  variables 
not  appearing  in  r  or  tb-  For  every  pair  of  locations  £1,^2  £  L,  we  will  now  trans¬ 
form  t(£  1,^2)  to  the  equivalent  3E  •  (te(£  1,^2)  A /\  E)  such  that  the  variables  in  E 
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only  appear  negatively  in  7£.  We  refer  to  E  as  assumptions  following  SAT  termi¬ 
nology  [  ].  Note  that  dropping  some  assumptions  from  the  conjunction  /\  E  results 

in  an  abstract  transition  relation,  i.e.,  |=  t(G,A)  ==>•  3E  •  i,£2)  A  /\  E j  for 

E  C  E.  We  write  r(E)  to  denote  the  resulting  abstract  transition  relation.  So,  we 
have  r  f  (E). 

Note  that  f  (E)  can  be  obtained  from  Ty  by  substituting  the  assumptions  in  E  by 
T  and  the  rest  of  the  assumptions  by  _L,  i.e.,  f  (E)  =  7e[E  ■<—  T,  E\E  ■<—  _L],  The  only 
abstractions  of  P  we  consider  are  the  ones  which  abstract  r  and  keep  tq  unchanged. 
That  is,  every  abstraction  P  of  P  is  such  that  P  -<i(i  P  with  Tp  =  t (E)  A  Tb  for  some 
E  C  E.  Given  E  C  E,  we  denote  the  corresponding  abstraction  as  P( E). 

Under-approximations.  Given  an  abstraction  P( E)  for  a  subset  of  assumptions 
E  C  E,  an  under-approximation  is  obtained  by  constraining  the  bound  variables  in 
B.  In  particular,  an  under- approximation  U (E,  bvals )  for  a  total  map  bvals  :  B  — >  N 
from  B  to  natural  numbers  is  obtained  by  strengthening  tb  with  the  constraints 
A b(zB  b  —  bvals(b),  for  every  pair  of  locations.  We  denote  the  strengthening  of  rB  by 
TB{bvals).  So,  the  under-approximation  U(E,  bvals)  satisfies  U{Yi,bvals)  Pi(i  P(E), 
with  the  transition  relation  t (E)  A  Tb  (bvals). 

Solve.  To  implement  Solve  (see  Fig.  3.7)  we  first  transform  Ux ,  the  restriction 
of  the  current  under-approximation  U  to  the  invariants  X,  to  Horn-SMT  [  i],  which 
essentially  encodes  the  verification  conditions  of  the  program  by  using  predicate 
variables  to  denote  the  unknown  invariants.  The  Horn-SMT  problem  is  then  passed 
to  the  tool  Z3  [  ].  While  Z3  is  primarily  an  SMT  solver,  it  also  has  the  capability 

of  solving  Horn-SMT  |  |.  Note  that,  in  presence  of  an  oracle  for  the  5-theory  Th 
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Global 


Invars 


Etj  =►  A  tb(4^).  4  A  e  i  (i) 

TV,  =►  V,  £i,i,  4  e  L  (2) 

(Vi  Eij)  T,  ii£L,<p&  1(G)  (3) 

TV,  =►  <p',  Ii&L,<p€  1(1,)  (4) 


Local 


Lemmas 

A^ei,ve7rpi)  ==’> 

((VjE«)  =*■  *>)) 

(5) 

.  (jy,  =»  iff)) 

(6) 

Assump.  Lits 

AetV,  i  €  L,  ifi  G  7 t(I) 
f  G  L,  ip  e  7r(£) 

(7) 

(8) 

Concrete 

E 

(9) 

Bound  Vais 

b  <  bvals(b),  b  £  B 

(10) 

Figure  3.11:  Constraints  used  in  our  implementation  of  Spacer. 


(e.g.,  linear  arithmetic),  U  is  decidable  as  the  length  of  a  feasible  path  is  bounded. 
However,  Z3  also  has  heuristics  for  solving  undecidable  problems.  See  Section  3.6  for 
a  comparison  between  Spacer  and  Z3.  Note  that  one  can  replace  Z3  with  any  other 
tool  that  solves  Horn-SMT  problems  (for  e.g.,  our  implementation  of  the  algorithm 
described  in  Chapter  2). 

Finally,  we  describe  how  to  implement  SPACER  efficiently  using  an  incremental 
SMT  solver  where  constraints  can  be  dynamically  added  or  retracted  for  checking 
satisfiability  of  multiple  instances.  We  implement  the  routines  of  SPACER  in  Fig.  3.7 
by  maintaining  a  set  of  constraints  C.  At  a  high  level,  there  are  two  types  of  con¬ 
straints,  as  shown  in  Fig.  3.11.  The  Global  constraints  are  global  to  all  the  routines 
and  C  is  updated  whenever  a  new  global  constraint  is  inferred  by  a  routine.  The 
Local  constraints  are  local  to  a  routine  which  are  added  to  or  retracted  from  C  as 
needed.  We  will  explain  the  various  constraints  below. 
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The  global  constraints  labeled  Trans  encode  the  transition  relation  of  P  using 
fresh  Boolean  variables  E%3  and  N%  for  transitions  and  locations,  respectively.  The 
intended  meaning  of  the  Boolean  variables  Ei3  and  iVj  is  as  follows:  (a)  setting  E%  3  to 
true  enables  the  transition  from  the  location  l ,  to  the  location  ip  also  implying  that 
the  current  location  is  £j,  and  (b)  setting  Nt  to  true  means  that  the  next  location  is  £j. 
The  constraints  in  (1)  (see  the  figure)  encode  Tp  while  leaving  out  the  assumptions 
in  E.  So,  choosing  an  abstraction  of  P  amounts  to  adding  a  subset  of  the  Boolean 
variables  in  E  as  additional  constraints.  The  constraints  in  (2)  enforce  that  a  location 
is  reachable  only  via  one  of  its  incoming  edges. 

The  global  constraints  labeled  Invars  encode  the  currently  known  invariants.  In 
order  to  specify  that  the  invariants  hold  before  and  after  a  transition,  we  encode 
the  invariants  in  terms  of  both  current-state  variables  (3)  and  next-state  variables 
(4).  To  identify  that  the  current  location  is  the  antecedent  in  (3)  specifies  that  at 
least  one  outgoing  transition  from  li  is  enabled.  Similarly,  to  identify  that  the  next 
location  is  £*,  iV*  is  used  as  the  antecedent  in  (4). 

For  a  set  of  Boolean  literals  A ,  let  Sat(C,  *4.)  be  a  function  that  checks  whether 
CllA  is  satisfiable,  and  returns  either  a  satisfying  assignment  or  an  unsat  core  A  C  A 
such  that  CUA  is  unsat isfiable.  Modern  SAT  and  SMT  solvers,  including  Z3,  support 
this  functionality  and  the  Boolean  literals  in  A  are  called  assumption  literals  [  ]. 

The  local  constraints  are  explained  along  with  the  implementations  of  the  various 
routines  below. 

InitAbs.  is  implemented  by  choosing  P  as  the  initial  abstraction,  i.e.,  the  initial 
subset  E  of  E  is  E  itself. 
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InitUnder.  is  implemented  by  first  initializing  bvals  to  Xb  E  B  ■  0  and  then 
choosing  U( E,  bvals )  as  the  initial  under-approximation. 

EXTRACTlNVS  is  implemented  by  EXTRACTlNVSlMPL  shown  in  Fig.  3.12.  Let  i r 
be  a  safety  proof  of  the  current  under-approximation.  EXTRACTlNVlMPL  extracts 
a  Maximal  Inductive  Subset  (MIS)  of  the  formulas  given  by  n  w.r.t.  the  concrete 
transition  relation  r  A  Tb  of  P ,  which  we  explained  intuitively  in  Section  3.2.  To 
concretize  the  transition  relation,  we  first  add  the  constraints  under  Concrete  in 
Fig.  3.11  to  C,  i.e.,  we  add  all  the  assumptions  in  E.  Then,  we  add  the  constraints 
under  Lemmas  to  C  which  encode  the  formulas  given  by  n  over  current  and  next-state 
variables  guarded  by  fresh  Boolean  variables  Ae,<p  and  for  every  location  I  and 
formula  tp  G  7t(£).  The  negation  in  (6)  is  used  to  encode  the  negation  of  invariance 
(Definition  4)  so  that  we  can  use  SAT  solving  to  check  validity.  We  simulate  the 
greatest  fixed-point  computation  shown  on  lines  21-22  in  Fig.  3.7  by  iteratively 
enabling  and  disabling  these  Boolean  variables  as  follows. 

The  MIS  of  7 r  corresponds  to  the  maximal  subset  /  C  such  that  CU/U 

{->3^  |  Ai)ip  1}  is  unsatishable.  Intuitively,  /  selects  a  subset  of  the  formulas  from 
7i(£)  over  the  current-state  variables,  for  every  location  £,  that  are  together  invariant 
(Definition  4).  Now,  in  order  to  disable  every  other  formula  over  the  next-state 
variables,  we  also  need  to  assert  ->£yjV,  where  p  is  a  formula  that  is  not  invariant.  I  is 
computed  by  ExtractInvsImpl  in  Fig.  3.12.  Each  iteration  of  ExtractInvsImpl 
refines  the  set  of  formulas  by  eliminating  the  ones  that  fail  the  invariance  check  when 
the  current  set  of  formulas  is  assumed  to  hold  of  the  current-state  variables  (according 
to  Definition  4).  This  can  be  accomplished  by  computing  the  least  subset  of  the 
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^  variables  to  disable,  given  the  current  subset  of  the  AetV>  variables,  where  the 
minimality  ensures  that  only  the  formulas  that  fail  the  invariance  check  are  removed. 
We  use  Minimal  Unsatisfiable  Subset  (MUS)  to  denote  such  a  subset. 

Suppose  we  are  interested  in  computing  the  minimal  subset  of  a  set  V  of  literals 
such  that,  together  with  another  fixed  set  T  of  literals,  the  constraints  in  C  are 
unsatisfiable.  There  are  several  choices  for  implementing  such  a  MUS  computation. 
A  naive  approach  is  to  simply  call  Sat(C,  T  U  V)  and  hope  that  the  unsat  core 
returned  by  the  SAT/SMT  solver  is  minimal.  However,  in  our  particular  case,  this 
is  guaranteed  to  fail  for  the  following  reason.  The  set  V  corresponds  to 
and  the  DPLL-style  search  strategy  employed  by  present  day  SAT/SMT  solvers 
works  by  first  setting  all  the  assumption  literals  to  true.  Given  that  setting  all 
these  assumption  literals  to  true  makes  the  constraint  in  (6)  unsatisfiable,  the  solver 
immediately  deduces  _L  and  returns  the  entire  set  as  the  unsat  core.  For  this  reason, 
we  use  an  alternative  MUS  computation  using  the  routine  Mus  in  Fig.  3.12,  which 
employs  a  bottom-up  iterative  strategy.  However,  note  that  the  minimality  of  the 
output  of  this  routine  depends  on  the  minimality  of  the  SAT  assignments  obtained 
on  line  10  in  the  figure.  That  is,  it  is  possible  that  a  literal  from  V  assigned  to  false 
by  the  model  on  line  10  is  actually  a  dont-care. 

Going  back  to  the  routine  EXTRACTlNVSlMPL,  M  on  line  4  corresponds  to  the 
cumulative  set  of  formulas  that  fail  the  invariance  check  and  X,  on  line  5,  corresponds 
to  all  the  other  formulas. 

Pba  finds  a  subset  of  assumptions  Si  C  E  such  that  ^f/(Si,  bvals)^j  is  safe  with 
the  same  proof  tt  of  the  current  under- approximation.  As  above,  we  first  add  the 
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ExtractInvsImpl(C,  {At^e#,  {Be,<p}e,<p) 
M  :=  0,  X  :=  {At,v}t,<pi  Y  :=  {-> 

T  :=  X 

while  (5  :=  Mus (C,T,Y))  ±  0  do 
M  :=  M  U  S,  Y  :=Y\  M 
X  :=  {At,v  |  ^Be,v  e  r} 

T  :=  XUM 


7  return  A' 


Mus(C,  T,  V) 
i?  :=  0 

while  Sat(C,TU-R)  do 

to  :=  GetModel(C,  T  U  -R) 
i?  :=  i?  U  {i;  G  V  |  to[i>]  =  false} 


12 


return  i? 


Figure  3.12:  Our  implementation  of  ExtractInvs  of  Fig.  3.7. 


constraints  under  Lemmas  in  Fig.  3.11  to  C  which  encodes  the  formulas  given  by 
7r  over  current  and  next-state  variables  guarded  by  fresh  Boolean  variables.  Then, 
the  constraints  under  Bound  Vais  in  Fig.  3.11  are  added  to  C  to  encode  the  under¬ 
approximation.  This  reduces  the  check  for  whether  the  map  tt  is  a  safety  proof 
to  that  of  unsatisfiability  of  a  formula.  Finally,  Sat(C,  £  U  {Ae!V>}e,<p  U  {Be,tp}e,<p)  is 
invoked.  As  tt  is  a  safety  proof  of  the  under-approximation,  this  must  result  in  an 
unsat  core.  Projecting  the  core  onto  £  gives  us  the  desired  £i  C  £  which  identifies 
the  new  abstraction  and,  together  with  the  current  bvals,  the  corresponding  new 
under- approximation.  The  minimality  of  £]  depends  on  the  algorithm  for  extracting 
an  unsat  core,  which  is  part  of  the  SMT  engine  of  Z3  in  our  case.  In  practice, 
we  make  iterative  SAT  calls  with  the  current  subset  of  £  in  place  of  £,  until  the 
returned  unsat  core  is  the  same  as  the  previous  subset  of  assumptions.  Note  that,  as 
we  treat  {AeiV,}itV,  and  as  assumption  literals  as  well,  the  SAT/SMT  solver 
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can  ignore  any  redundant  formulas  in  the  proof  n  and  such  redundancy  is  quite 
possible  in  practice. 

NextUnder.  Given  the  current  valuation  bvals  and  the  new  abstraction  E,  this 
routine  returns  U( E,  A b  E  B  ■  bvalsib)  +  1). 

Cegar  and  IsFeasible.  Let  ^f/( E,  bvals)^j  be  unsafe  with  a  counterexample 
.  We  create  a  new  set  of  constraints  C ^  corresponding  to  the  unrolling  of  Tv  A  Tb 
along  the  control  path  of  ^  and  check  Sat(C<^,E).  If  this  returns  a  satisfiable 
assignment,  the  counterexample  is  feasible  in  P  and  the  assignment  is  used  to  find 
a  counterexample  to  safety  in  P.  Otherwise,  we  obtain  an  unsat  core  Ej  C  E  and 
refine  the  abstraction  to  EUEi. 

We  conclude  the  section  with  a  discussion  of  the  implementation  choices.  The 
above  implementation  of  NextUnder  increments  all  bounding  variables  uniformly. 
An  alternative  is  to  increment  the  bounds  only  for  the  loops  for  which  the  formu¬ 
las  in  the  current  proof  tt  fail  to  be  invariant  (e.g.,  [5,  89]).  However,  we  leave  the 
exploration  of  such  strategies  for  future.  Our  use  of  Z3  is  sub-optimal  as  each  call 
to  SOLVE  requires  constructing  a  new  Horn-SMT  problem.  This  incurs  an  unnec¬ 
essary  pre-processing  overhead  that  can  be  eliminated  by  a  tighter  integration  with 
Z3.  For  Pba  and  ExtractInvs,  we  use  a  single  SMT-context  with  a  single  copy  of 
the  transition  relation  of  the  program  (without  unrolling  it)  by  means  of  the  Global 
constraints  mentioned  above.  This  SMT-context  is  preserved  across  iterations  of 
Spacer.  Constraints  specific  to  a  routine  are  added  and  retracted  using  the  in¬ 
cremental  solving  API  of  Z3.  This  is  vital  for  good  performance  in  practice.  For 
CEGAR  and  IsFeasible,  we  unroll  the  transition  relation  of  the  program  along  the 
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control  path  of  the  counterexample  trace  returned  by  Z3.  We  experimented  with  an 
alternative  implementation  that  instead  validates  each  individual  step  of  the  (sym¬ 
bolic)  counterexample  using  the  same  global  context  as  Pba.  While  this  made  each 
refinement  step  faster,  it  increased  the  number  of  refinements,  becoming  inefficient 
overall. 


3.6  Experimental  Results 

We  have  a  prototype  implementation  of  SPACER  using  the  SMT  solver  Z3  [  |.  The 

implementation  and  complete  experimental  results  are  available  online.7 
Benchmarks.  We  used  the  C  program  benchmarks  of  the  Software  Verification 
Competition  2013  [  ].  As  our  tool  does  not  yet  handle  memory  related  properties, 
we  confined  ourselves  to  the  categories  of  systemc ,  product-lines ,  device- drivers- 64 
and  control- flow-integers.  All  the  benchmarks  are  available  on  the  competition  web¬ 
site  [  ].  We  give  a  brief  description  of  the  4  categories  below  (and  refer  the  reader  to 
the  competition  website  for  more  details): 

systemc :  these  are  derived  from  SystemC  programs  in  the  literature,  which  have 
been  transformed  to  pure  C  programs  by  incorporating  the  scheduler  into  the 
C  code. 

product-lines :  these  are  derived  from  a  research  project  for  integration  verification 
of  software  product  lines. 

device- drivers- 6 4-  these  are  derived  from  the  Linux  Driver  Verification  |  |  project 

7  http : //www. cs . emu. edu/~akomurav/projects/ spacer/home .  html 
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and  correspond  to  the  actual  Linux  kernel  code. 

control- flow-integers',  this  contains  programs  whose  safety  properties  depend  mostly 
on  the  control-flow  structure  and  integer  variables,  taken  from  the  repositories 
of  the  tools  BLAST  [70]  and  CPAchecker  [1], 

As  mentioned  in  the  previous  section,  we  used  the  implementation  of  the  al¬ 
gorithm  GPDR  [74]  in  Z3  for  the  Solve  step  in  each  iteration  of  Spacer.  The 
front-end,  which  translates  a  C  program  to  the  Horn-SMT  format  of  Z3,  is  based 
on  the  tool  UFO  [8].  The  encoding  in  fforn-SMT  only  uses  the  theory  of  Linear 
Rational  Arithmetic.  All  experiments  were  carried  out  on  an  Intel®  Core™2  Quad 
CPU  of  2.83GHz  and  4GB  of  RAM.  The  resource  limits  were  set  to  15  minutes  of 
time  and  2GB  of  memory. 

Overall,  there  are  1,990  benchmarks  (1,591  marked  SAFE,  and  399  marked  UN¬ 
SAFE);  1,382  are  decided  by  the  front-end  of  UFO  that  uses  common  compiler 
optimizations.  This  left  608  benchmarks  (231  SAFE,  and  377  UNSAFE).  To  evalu¬ 
ate  the  advantage  of  abstractions,  we  also  ran  (the  implementation  of  GPDR  in)  Z3 
by  itself  on  the  benchmarks  and  compared  with  SPACER. 

For  the  UNSAFE  benchmarks,  Fig.  3.13  shows  a  scatter  plot  for  the  369  bench¬ 
marks  verified  in  both  settings,  with  and  without  abstraction;  of  the  remaining  8 
benchmarks,  6  are  unverified,  and  2  are  verified  without  abstraction  but  not  by 
SPACER.  Note  that,  even  though  abstraction  did  not  help  for  these  benchmarks,  the 
properties  are  easy,  with  Spacer  needing  at  most  3  minutes  each.  We  will  show 
later  in  the  section  that  a  traditional  CEGAR  approach  (without  PBA)  can  be  even 
worse. 
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Figure  3.13:  Advantage  of  abstractions  (Spacer  vs.  Z3)  for  UNSAFE  benchmarks. 
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Figure  3.14:  Advantage  of  abstractions  (Spacer  vs. 


Z3)  for  SAFE  benchmarks. 
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Figure  3.15:  The  best  of  the  three  variants  of  Spacer  against  Z3  for  SAFE  benchmarks. 
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For  the  SAFE  benchmarks,  see  Fig.  3.14  for  a  scatter  plot  comparing  model 
checking  with  and  without  abstraction  (i.e.,  Spacer,  and  Z3,  respectively).  176 
benchmarks  are  verified  in  under  a  minute  in  both  settings  (see  the  dense  set  of 
triangles  in  the  lower  left  corner  of  the  figure).  For  these  benchmarks,  the  difference 
in  runtime  is  not  significant  enough  to  be  meaningful.  We  analyze  the  rest  of  the 
results  below. 


Detailed  Results.  Table  3.1  shows  the  experimental  results  on  the  42  safe  bench¬ 
marks  verified  by  either  tool  and  needing  more  than  a  minute  of  running  time.  The  t 
columns  under  Z3  and  SPACER  show  the  running  times  in  seconds  with  ‘TO’  indi¬ 
cating  a  time-out  and  a  ‘MO’  indicating  a  mem-out.  The  best  times  are  highlighted 
in  bold.  The  corresponding  scatter  plot  in  Fig.  3.14  shows  that  the  results  are  mixed 
for  a  time  bound  of  300  seconds  (5  minutes).  But  beyond  5  minutes,  abstraction 
really  helps  with  many  benchmarks  verified  by  SPACER,  when  Z3  runs  out  of  time 
(time-outs  are  indicated  by  diamonds  and  mem-outs  are  indicated  by  stars).  The 
couple  of  benchmarks  where  SPACER  runs  out  of  time  become  better  than  Z3  using  a 
different  setting,  as  discussed  later.  Overall,  abstraction  helps  for  hard  benchmarks. 
Furthermore,  in  elev_13_22,  elev_13_29  and  elev_13_30,  SPACER  is  successful 
even  though  Z3  runs  out  of  memory,  showing  a  clear  advantage  of  abstraction  (this 
corresponds  to  the  stars  in  the  far  right  of  Fig.  3.14).  Note  that  the  gcnr  example 
in  the  table  under  raise  is  from  Fig.  3.1. 

The  B  column  in  the  table  shows  the  final  values  of  the  loop  bounding  variables 
under  the  mapping  bvals,  i.e.,  the  maximum  number  of  loop  iterations  (of  any  loop) 
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that  was  necessary  for  the  final  safety  proof.  Surprisingly,  they  are  very  small  in 
many  of  the  hard  instances  in  systemc  and  product-lines  categories. 

Columns  df  and  am  show  the  sizes  of  the  final  and  maximal  abstractions,  re¬ 
spectively,  measured  in  terms  of  the  number  of  the  original  constraints  used.  Note 
that  this  only  corresponds  to  the  syntactic  abstraction  (see  Section  3.4).  The  final 
abstraction  computed  by  SPACER  is  very  aggressive.  Many  constraints  are  irrelevant 
(given  the  computed  invariants)  with  often,  more  than  50%  of  the  original  constraints 
abstracted  away.  Finally,  the  difference  between  aj  and  am  is  insignificant  in  all  of 
the  benchmarks. 

An  alternative  approach  to  Pba  is  to  restrict  the  abstraction  to  state- variables  by 
allowing  some  of  the  variables  to  take  next-state  values  non-deterministically  without 
any  constraints,  similar  to  the  work  by  Vizel  et  al.  [105]  in  the  context  of  hardware 
verification.  This  was  especially  effective  for  ssh  and  ssh- simplified  categories  -  see 
the  entries  marked  with  under  column  t. 

An  alternative  implementation  of  CEGAR  is  to  concretize  the  under-approximation 
(by  refining  E  to  E)  whenever  a  spurious  counterexample  is  found.  This  is  analogous 
to  Proof-Based  Abstraction  (PBA)  [91]  in  hardware  verification.  Run-time  for  PBA 
and  the  corresponding  final  values  of  the  bounding  variables  are  shown  in  columns 
tp  and  Bp  of  Table  3.1,  respectively.  While  this  results  in  more  time-outs,  it  is  sig¬ 
nificantly  better  in  14  cases  (see  the  entries  marked  with  ‘f’  under  column  tp),  with 
6  of  them  comparable  to  Z3  and  2  (viz.,  toy  and  elev_l_3l)  significantly  better 
than  Z3. 
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Benchmark 

Z3 

Spacer 

t 

(sec) 

t 

(sec) 

B 

af 

(%) 

(%) 

tp 

(sec) 

Bp 

systemc 

pipeline 

224 

120 

4 

33 

33 

249 

4 

tk_ring_06 

64 

48 

2 

59 

59 

65 

2 

tk_ring_07 

69 

120 

2 

59 

59 

f67 

2 

tk_ring_08 

232 

158 

2 

57 

57 

358 

2 

tk_ring_09 

817 

241 

2 

59 

59 

266 

2 

mem_slave_l 

536 

430 

3 

24 

34 

483 

2 

toy 

TO 

822 

4 

32 

44 

f460 

4 

pc_sf if o_2 

73 

137 

2 

41 

41 

TO 

- 

product-lines 

elev_13_21 

TO 

174 

2 

7 

7 

TO 

- 

elev_13_22 

MO 

336 

2 

9 

9 

624 

4 

elev_13_23 

TO 

309 

4 

6 

14 

TO 

- 

elev_13_24 

TO 

591 

4 

9 

9 

TO 

- 

elev_13_29 

MO 

190 

2 

6 

10 

TO 

- 

elev_13_30 

MO 

484 

3 

11 

13 

TO 

- 

elev_13_31 

TO 

349 

4 

8 

17 

TO 

- 

elev_13_32 

TO 

700 

4 

9 

9 

TO 

- 

elev_l_21 

102 

136 

11 

61 

61 

161 

11 

elev_l_23 

101 

276 

11 

61 

61 

fl40 

11 

elev_l_29 

92 

199 

11 

61 

62 

f77 

11 

elev_l_31 

127 

135 

11 

62 

62 

f92 

11 

elev_2_29 

18 

112 

11 

56 

56 

f26 

11 

elev_2_31 

16 

91 

11 

57 

57 

f22 

11 

ssh 

s3_clnt_3 

109 

*90 

12 

13 

13 

73 

12 

s3_srvr_l 

187 

43 

9 

18 

18 

661 

25 

s3_srvr_2 

587 

*207 

14 

3 

7 

446 

15 

s3_srvr_8 

99 

49 

13 

18 

18 

TO 

- 

s3_srvr_10 

83 

24 

9 

17 

17 

412 

21 

s3_srvr_13 

355 

*298 

15 

8 

8 

461 

15 

s3_clnt_2 

34 

*124 

13 

13 

13 

f95 

13 

s3_srvr_12 

21 

*64 

13 

8 

8 

54 

13 

s3_srvr_14 

37 

*141 

17 

8 

8 

f91 

17 

s3_srvr_6 

98 

TO 

- 

- 

- 

1300 

25 

s3_srvr_ll 

270 

896 

15 

14 

18 

831 

13 

s3_srvr_15 

309 

TO 

- 

- 

- 

TO 

- 

s3_srvr_16 

156 

*263 

21 

8 

8 

1159 

21 

ssh- simplified 

s3_srvr_3 

171 

130 

11 

21 

21 

116 

12 

s3_clnt_3 

50 

*139 

12 

17 

22 

1104 

13 

s3_clnt_4 

15 

*76 

12 

22 

22 

56 

13 

s3_clnt_2 

138 

509 

13 

26 

26 

1145 

13 

s3_srvr_2 

148 

232 

12 

16 

23 

222 

15 

s3_srvr_6 

91 

TO 

- 

- 

- 

1272 

25 

s3_srvr_7 

253 

398 

10 

20 

26 

764 

10 

misc 

gcnr 

TO  | 

|  56 

26 

8! 

95  | 

|  50 

25 

Table  3.1:  Comparison  of  Z3  and  Spacer,  t  and  tp  are  running  times  in  seconds;  B  and  Bp  are  the 
final  values  of  the  bounding  variables;  aj  and  am  are  the  fractions  of  assumption  variables  in  the 
final  and  maximal  abstractions,  respectively. 
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See  Fig.  3.15  for  a  scatter  plot  using  the  best  running  times  for  Spacer  of  all  the 


three  variants  described  above. 


Advantage  of  PBA.  To  better  understand  the  effect  of  Proof-Based  Abstraction 
(PBA),  we  ran  Spacer  with  PBA  disabled  and  choosing  the  coarsest  abstraction  as 
our  initial  abstraction.  Note  that  this  is  the  traditional  CEGAR  approach.  Fig.  3.16 
and  3.17  show  the  scatter  plots  for  the  same  benchmarks  as  above  comparing  CEGAR 
with  and  without  PBA  (PBA  +  CEGAR  is  essentially  Spacer  as  described  until 
now).  These  plots  show  that,  in  many  cases,  PBA  results  in  quite  a  significant 
improvement  over  traditional  CEGAR-based  abstraction  refinement.  We  believe 
that  this  is  because  PBA  results  in  abstractions  that  are  relevant  (proof-based)  and 
precise  (due  to  invariants).  The  runtimes  shown  in  the  plot  correspond  to  the  best 
of  the  abstraction  mechanisms  with  and  without  restricting  to  the  state- variables  as 
mentioned  above. 

Table  3.2  also  shows  the  number  of  abstraction  refinement  iterations  and  the 
final  abstraction  size  with  and  without  PBA  on  the  8  safe  examples  for  which  both 
approaches  terminate  and  CEGAR  takes  more  than  5  minutes.  Despite  the  fact  that 
adding  PBA  results  in  seemingly  more  amount  of  work  (for  invariant  extraction,  re¬ 
abstraction  after  each  iteration  of  Spacer,  etc.)  to  maintain  coarser  abstractions 
(compare  columns  am  and  a),  the  number  of  abstraction  refinement  iterations  is 
increased  only  in  a  couple  of  examples.  Interestingly,  the  value  of  the  bounding 
variables  for  which  both  approaches  terminate  is  the  same  for  all  the  examples. 

We  conclude  this  section  by  comparing  our  results  with  UFO  [8]  —  the  win- 
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Figure  3.16:  Advantage  of  PBA  for  SAFE  benchmarks. 
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Figure  3.17:  Advantage  of  PBA  for  UNSAFE  benchmarks. 

ner  of  the  4  benchmark  categories  at  SV-COMP’13.  The  competition  version  of 
UFO  runs  several  engines  in  parallel,  including  engines  based  on  Abstract  Interpre¬ 
tation,  Predicate  Abstraction,  and  SMT-based  model  checking  with  Interpolation. 
UFO  outperforms  Spacer  and  Z3  in  ssh  and  product-lines  categories  by  an  order 
of  magnitude.  These  benchmarks  seem  to  be  easier  for  Abstract  Interpretation  and 
Predicate  Abstraction  used  in  UFO  but  this  needs  more  investigation.  Even  so,  note 
that  SPACER  finds  really  small  abstractions  for  these  categories  upon  termination. 
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Benchmark 

PBA  +  CEGAR 

CEGAR 

t 

(sec) 

B 

liters 

af 

(%) 

(%) 

t 

(sec) 

B 

if  iters 

a 

(%) 

systemc 

tk_ring_09 

242 

2 

n 

59 

59 

457 

2 

12 

59 

mem_slave_l 

430 

3 

59 

24 

34 

549 

3 

55 

36 

product-lines 

elev_13_30 

484 

3 

60 

11 

13 

472 

3 

65 

14 

elev_l_21 

136 

11 

17 

61 

61 

525 

11 

18 

64 

elev_l_23 

276 

11 

16 

61 

61 

772 

11 

17 

63 

elev_l_29 

199 

11 

17 

61 

62 

759 

11 

17 

62 

elev l 31 

134 

11 

17 

62 

62 

353 

11 

17 

63 

ssh- simplified 

s3_srvr_2 

232 

12 

22 

12 

16 

*592 

12 

17 

22 

Table  3.2:  Analyzing  the  effect  of  PBA  on  some  hard  examples,  t  denotes  the  running  time;  B 
is  the  final  value  of  the  bounding  variables;  liters  denotes  the  number  of  abstraction  refinement 
iterations;  a /  and  am  are  the  fractions  of  assumption  variables  in  the  final  and  maximal  abstractions 
with  PBA;  a  is  the  fraction  of  the  assumption  variables  in  the  final  abstraction  (which  is  also  the 
maximal)  without  PBA. 


However,  in  the  systemc  category  both  Spacer  and  Z3  perform  better  than  UFO 
by  verifying  hard  instances  (e.g.,  tk_ring_08  and  tk_ring_09)  that  are  not  verified 
by  any  tool  in  the  competition.  Moreover,  SPACER  is  faster  than  Z3,  in  general,  as 
shown  above.  Thus,  while  SPACER  itself  is  not  the  best  tool  for  all  benchmarks,  it 
is  a  valuable  addition  to  the  state-of-the-art. 


3.7  Related  work 

The  most  prominent  approach  for  iteratively  checking  bounded  safety  is  to  combine 
BMC  with  Craig  Interpolation  [5,  88,  89].  Recently,  algorithms  for  incremental  BMC, 
together  with  interpolation,  have  also  been  proposed  [25,  34,  48,  7  ].  Although  our 
implementation  uses  Z3  (for  SOLVE),  which  is  based  on  an  incremental  algorithm,  it 
can  be  implemented  on  top  of  any  interpolation-based  solver. 

Proof-based  Abstraction  (PBA)  was  first  introduced  in  hardware  verification  to 
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leverage  the  power  of  SAT-solvers  to  focus  on  relevant  facts  [66,  91].  Over  the  years, 
it  has  been  combined  with  CEGAR  [12,  13],  interpolation  [13,  85],  and  PDR8  [  5],  all 
in  the  context  of  hardware  model  checking.  To  the  best  of  our  knowledge,  SPACER  is 
the  first  application  of  PBA  for  automatic  abstraction  refinement  in  software  model 
checking. 

Our  extraction  of  maximal  invariant  subsets  from  candidate  proofs  (of  bounded 
safety,  in  our  case)  is  similar  to  HOUDINI  [5  ]  and  is  used  in  several  other  algorithms 
(e.g.,  [90]).  As  in  SPACER,  Jain  et  al.  have  also  used  program  invariants  to  obtain 
precise  abstractions  in  the  context  of  predicate  abstraction  [  56] . 

The  work  of  Vizel  et  al.  [105],  in  hardware  verification,  that  extends  PDR  with 
abstraction  is  the  closest  to  ours.  However,  SPACER  is  not  tightly  coupled  with  PDR. 
Moreover,  Spacer  allows  for  a  rich  space  of  abstractions,  whereas  Vizel  et  al.  limit 
themselves  to  state  variable  abstraction. 

Finally,  the  tool  UFO  [5,  6]  also  uses  abstraction,  but  in  an  orthogonal  way. 
UFO  uses  abstraction  to  guess  the  depth  of  unrolling  (plus  useful  invariants),  BMC 
to  detect  counterexamples,  and  interpolation  to  synthesize  safe  invariants. 

3.8  Conclusion 

In  this  chapter,  we  presented  the  SPACER  algorithm  that  combines  Proof-Based  Ab¬ 
straction  (PBA)  with  CounterExample  Guided  Abstraction  Refinement  (CEGAR) 
for  verifying  safety  properties  of  sequential  programs.  To  our  knowledge,  this  is  the 

first  application  of  PBA  to  software  verification.  Our  abstraction  technique  com- 
8PDR  stands  for  Property  Directed  Reachability. 
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bines  localization  with  invariants  about  the  program.  It  is  interesting  to  explore 
alternatives  for  such  a  semantic  abstraction. 

While  our  presentation  is  restricted  to  non-recursive  sequential  programs,  the 
technique  can  be  adapted  to  solving  the  more  general  Horn  Clause  Satisfiability 
problem  and  extended  to  verifying  recursive  and  concurrent  programs  [63]. 

We  have  implemented  SPACER,  using  Z3  and,  in  particular,  its  GPDR  engine. 
Our  implementation  is  only  an  early  prototype  and  is  not  heavily  optimized  nor  it  is 
tightly  integrated  with  Z3.  Nonetheless,  the  experimental  results  on  4  categories  of 
the  2nd  Software  Verification  Competition  show  that  Spacer,  improves  on  both  Z3 
and  the  state-of-the-art. 

The  results  presented  in  this  chapter  are  published  as  part  of  the  proceedings  of 
CAV  2013  [80]. 
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3. A  Transforming  a  Safety  Proof  of  P  to  that  of  P 


Let  P  =(L,  £°,  £e,  V,  r)  be  the  input  program  and  let  P  be  obtained  by  the  trans¬ 
formation  described  in  Section  3.5  which  adds  code  to  count  the  number  of  loop 
iterations.  The  following  lemma  shows  how  to  translate  a  safety  proof  of  P  to  that 
of  P.  Given  a  sentence  p  over  the  signature  S  U  VU  C  U  P,  we  write  VB  >  0 ,C  >  0  -p 
to  mean  VB  U  C  ■  ((A*gbuCx  —  0)  Pj- 

Lemma  10.  If  V  is  a  safety  proof  of  P,  then  7r  =  \£  ■  {VB  >  0,C  >  0  ■  p  \  p  £  7f(£)} 
is  a  safety  proof  of  P . 

Proof.  We  will  first  show  that  i r  is  safe.  We  have 

y\  7r(£e)  =  f\  VB  >  0,C  >  0-<p  =  VB  >  0,C  >  0-  /\  ip. 

ipE.Tr(ie)  (f€.  Tr(ie) 

Given  that  V  is  safe,  A^g-s-^)  V  = =>•  -L  and  hence,  f\7r(£e)  ==>■  _L. 

We  will  next  show  that  7r  is  an  invariant  map.  As  T  f\V{i0),  T  p  for 
every  p  £  7r(l°)  and  hence,  T  VB  >  0,(7  >  0  •  p.  So,  T  =>■  /\ti(I0). 

Assume  an  arbitrary  5-structure  (that  is  also  a  model  of  Th).  Let  s,  s'  be  a 
pair  of  current  and  next  states  satisfying  A7r(A)  A  r{£i,£j)  for  some  £% ,  £j  £  L.  We 
need  to  prove  that  VB  >  0,C  >  0  •  p  is  true  for  s',  for  every  p  £  j t(£j).  Let  b\d 
be  arbitrary  non-negative  values  for  B,  C,  respectively.  One  can  easily  show  that 
rB(£i,£j)  is  invertible  for  non-negative  values  of  the  post-variables  and  let  b,c  be  the 
values  of  the  corresponding  pre-variables.  Now,  for  b,  c  and  s,  we  know  that  f\  7r(0) 
is  true.  Given  that  V  is  a  proof  of  P,  it  follows  that  p  is  true  for  b',  c'  and  s'.  □ 
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Chapter  4 


Probabilistic  Systems  and  Simulation 

4.1  Introduction 

We  will  now  consider  safety  of  systems  with  probabilistic  behavior.  As  mentioned  in 
Chapter  1,  such  systems  are  increasingly  used  for  a  variety  of  applications  and  it  is 
important  to  be  able  to  efficiently  verify  their  correctness.  In  particular,  we  consider 
the  problem  of  checking  strong  simulation  conformance  between  two  probabilistic 
transition  systems,  an  implementation  and  a  specification.  In  this  chapter,  we  will 
describe  the  basic  definitions  and  algorithms  which  are  used  in  later  chapters. 

We  start  with  defining  our  notion  of  a  probabilistic  transition  system  in  Sec¬ 
tion  4.2.  We  will  then  define  the  conformance  relation  we  are  interested  in,  along 
with  several  key  properties  of  the  relation,  in  Section  4.3.  Following  that,  we  will 
describe  several  algorithms  for  monolithic  verification  of  the  conformance  relation 
in  Section  4.4,  which  include  a  new  reduction  to  SMT  and  a  specialized  algorithm 


113 


for  tree-shaped  transition  systems.  As  noted  in  Chapter  1,  when  the  conformance 
fails  to  hold  between  two  probabilistic  transition  systems,  there  is  no  existing  notion 
of  a  diagnostic  counterexample  which  explains  the  failure.  However,  this  turns  out 
be  an  essential  ingredient  for  automating  compositional  reasoning  as  we  will  see  in 
Chapters  5  and  6.  Section  4.5  describes  our  characterization  of  a  counterexample 
to  simulation  conformance,  including  several  key  properties  and  an  algorithm  for 
obtaining  a  counterexample.  Finally,  we  will  define  the  notion  of  parallel  composi¬ 
tion  between  transition  systems  and  show  the  soundness  and  completeness  of  the 
assume- guarantee  inference  rule  ASym,  also  mentioned  in  Chapter  1. 


4.2  Probabilistic  Transition  Systems 

In  the  probabilistic  transition  systems  we  consider,  a  transition  from  a  state  leads 
to  a  discrete  probability  distribution  over  states.1  Thus,  given  a  finite,  non-empty 
set  S  of  states,  a  state  s  G  S,  and  a  label  a,  a  transition  is  a  triple  (s,  a,  p)  for  a 
discrete  probability  distribution  p  over  S.  We  use  s  A-  p  to  denote  such  a  transition 
and  Fig.  4.1  shows  an  example.  We  use  Dist(S)  to  denote  the  set  of  all  discrete 
probability  distributions  over  S.  For  p  G  Dist(S),  the  support  of  p,  denoted  Supp(p), 
is  defined  to  be  the  subset  of  S  where  each  state  has  a  non-zero  probability  under 
p ,  i.e.,  Supp(p)  =  {s  G  S  |  p(s )  >  0}.  For  X  C  S,  p(X)  stands  for 
We  use  Ss  to  denote  the  special  Dirac  distribution  on  s  G  S  where  Ss(s)  =  1  and 
5s(S\{S})  =  0. 

lrTo  emphasize,  one  can  perhaps  use  the  term  probabilistic  transition  but  we  avoid  the  adjective 
for  brevity. 
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0.1 


Figure  4.1:  An  example  transition  from  a  state  s  on  an  action  a  to  a  discrete  probability  distribution 
/i  over  the  states  u  and  v. 


Figure  4.2:  Two  Labeled  Probabilistic  Transition  Systems  L\  and  L2- 


We  use  the  following  definition  of  a  transition  system  to  represent  probabilistic 
systems. 

Definition  7.  A  Labeled  Probabilistic  Transition  System  (LPTS)  is  a  tuple  ( S ,  s°,  a, 
r)  for  a  finite  set  of  states  S  with  a  designated  start  state  s°,  a  finite  set  of  actions 
a,  and  a  finite  set  of  transitions  r  C  S  x  a  x  Dist(S'). 

An  LPTS  is  called  reactive  if  t  is  a  partial  function  from  S  x  a  to  Dist(S')7  i.e., 
r  allows  at  most  one  transition  on  a  given  action  from  a  given  state.  An  LPTS  is 
called  fully-probabilistic  if  r  is  a  partial  function  from  S  to  oxDist(S'),  i.e.,  r  allows 
at  most  one  transition  from  a  given  state. 

For  example,  Fig.  4.2  shows  two  example  LPTSes  L\  and  L2  where  the  start  states 
are  denoted  by  filled  circles.  As  the  example  shows,  we  allow  multiple,  possibly  11011- 
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Figure  4.3:  An  example  of  a  stochastic  tree. 

deterministic,  transitions  outgoing  from  a  given  state.2  Note  that  L\  is  reactive 
while  L 2  is  not  reactive  because  of  the  non-determinism  on  action  a  from  state  q. 
The  figure  also  shows  Dirac  distributions  on  actions  b  and  c. 

In  the  literature,  an  LPTS  is  also  called  a  simple  probabilistic  automaton  [102]. 
Similarly,  a  reactive  (fully-probabilistic)  LPTS  is  also  called  a  (Labeled)  Markov  De¬ 
cision  Process  ( Markov  Chain).  Also,  note  that  an  LPTS  with  all  the  distributions 
restricted  to  Dirac  distributions  is  the  classical  (non-probabilistic)  Labeled  Transi¬ 
tion  System  (LTS);  thus  a  reactive  LTS  corresponds  to  the  standard  notion  of  a 
deterministic  LTS. 

We  are  also  interested  in  LPTSes  with  a  tree  structure,  i.e. ,  the  start  state  is  not 
in  the  support  of  any  transition’s  distribution  and  every  other  state  is  in  the  support 
of  exactly  one  transition’s  distribution.  We  call  such  LPTSes  stochastic  trees  or 
simply  trees.  For  example,  Fig.  4.3  shows  a  stochastic  tree. 

2This  is  useful  for  high  level  modeling  where  multiple  probabilistic  behaviors  are  allowed.  More¬ 
over,  the  nature  of  counterexamples  to  strong  simulation  (see  Section  4.5)  and  the  algorithms  for 
compositional  reasoning  (see  Chapters  5  and  6)  do  not  simplify  if  non-determinism  is  disallowed. 
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1/3 
1/6 
1/6 

1/3 

Figure  4.4:  Two  discrete  probability  distributions,  y  over  S  =  {si,S2}  and  v  over  T  = 

and  a  binary  relation  R  between  S  and  T,  shown  using  dotted  arrows,  such  that  /i  v.  The 

labeling  along  the  /hedges  shows  a  weight  function  used  to  establish  the  relationship  C#. 

4.3  Strong  Simulation 

To  specify  correctness  of  a  probabilistic  system  represented  by  an  LPTS,  we  use  the 
notion  of  strong  simulation  conformance  with  respect  to  a  specification  LPTS.  This 
is  based  on  the  standard  definition  of  simulation  conformance  between  transition 
systems  (a  la  Milner  [93]).  Intuitively,  we  say  that  a  transition  system  A  is  simulated 
by  another  transition  system  B  if  one  can  exhibit  a  binary  relation  between  the 
states  of  A  and  B  such  that  from  every  related  state  pair,  there  exist  transitions  in 
A  and  B  to  states  that  are  also  related,  whenever  the  transition  in  A  is  feasible.  For 
probabilistic  systems,  however,  a  transition  leads  to  a  distribution  of  states  and  we 
need  an  appropriate  notion  for  a  related  distribution.  For  this,  we  use  the  following 
definition  by  Segala  and  Lynch  [102]: 

Definition  8  ([102]).  Let  S  and  T  be  two  non-empty  finite  sets,  R  C  S  x  T ,  and 
consider  distributions  /i  €  Dist(S)  and  v  G  Dist(T).  We  say  that  v  is  a  related 
distribution  of  //  with  respect  to  R,  denoted  /i  C#  u,  iff  there  is  a  weight  function 
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w  :  S  x  T  — >  [0, 1]  such  that 

1.  for  every  s  G  S,  p(s)  =  ^^rpUiisR), 

2.  for  every  t  G  T,  v(t)  =  ^2seSw(s,t), 

3.  for  every  s  G  S,t  G  T ,  w(s,t )  >  0  implies  sRt. 

Intuitively,  fi  v  if  the  probabilities  of  states  in  S  under  //  can  be  distributed  to 
related  states  in  T  under  R  (as  suggested  by  a  suitable  weight  function)  to  obtain  u. 
See  Fig.  4.4  for  an  illustration.  Here,  S  =  {si,s2},  T  =  {ti,  t2,  ^3},  3  and  v  are  the 
uniform  distributions  over  S  and  T,  and  R  relates  the  states  connected  by  dotted 
arrows.  The  figure  also  shows  a  weight  function  w  by  means  of  a  labeling  of  the 
.R-edges  by  numbers  (all  other  state  pairs  are  mapped  to  0  under  w).  It  is  easy  to 
check  that  w  satisfies  the  conditions  in  the  above  definition  which  shows  that  fi  v. 
Effectively,  w  distributes  the  probability  /i(s  1)  =  1/2  as  1/3  to  tx  and  1/6  to  t2,  and 
the  probability  ^(s2)  =  1/2  as  1/6  to  t2  and  1/3  to  I3,  resulting  in  the  distribution 
v.  Checking  reduces  to  checking  whether  the  maximum  flow  in  an  appropriate 
network  is  equal  to  1.0  [17].  Note  that  is  a  binary  relation  between  distributions, 
given  R.  Using  we  can  define  a  Milner-style  simulation  conformance  between 
LPTSes  as  follows. 

Definition  9  (Strong  Simulation  [  ]).  Let  L\  =  (Si,  s°,  «i,  rf)  and  L2  =  (S2,  s®,  a2, 

r2)  be  two  LPTSes.  R  C  Si  x  S2  is  a  strong  simulation  iff  for  every  si  G  Si  and 
s2  G  S2,  if  s±Rs2  then  the  following  holds:  for  every  a  G  «i  and  si  A  there  is  a 
p2  G  Dist(S2)  with  the  property  that  s2  A  p2  and  ji\  /i2. 

For  s\  G  Si  and  s2  G  S2,  we  say  that  s2  strongly  simulates  s\,  denoted  S\  P  s2, 
iff  there  is  a  strong  simulation  T  such  that  SiTs2.  L2  strongly  simulates  L\ ,  also 


118 
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Figure  4.5:  Showing  y  Qr  v  for  distributions  y  and  v  of  Fig.  4.2,  and  a  binary  relation  R  shown 
using  dotted  arrows.  The  labeling  on  the  i?-edges  denotes  the  weight  function  used  to  show 

denoted  Li  ■<  L2,  iff  the  start  state  of  L2  strongly  simulates  the  start  state  o/L1;  i.e., 

Si  f  5°. 

For  example,  in  Fig.  4.2,  Li  R  L2  can  be  shown  using  the  strong  simulation 
R  —  {( p,q ),  (s,  u),  ( s ,  v),  (' t ,  u)}.  In  particular,  the  start  states  p  and  q  are  related  by 
R  because  the  outgoing  transition  pA/i  is  simulated  by  the  transition  q  A  z/,  i.e., 
/i  IZR  v.  As  shown  in  Fig.  4.5,  the  latter  can  be  shown  using  the  weight  function 
w  :  Si  x  5*2  — y  [0,1]  where  w(s,u )  =  1/6,  w(s,v )  =  1/2,  w(t,u )  =  1/3  (s,  t,  u,  and 
v  are  as  shown  in  the  figure)  and  w  maps  every  other  state  pair  to  0.  Note  that  q 
has  non-determinism  on  action  a,  however,  /i  5q  as  neither  s  nor  t  is  related  to  q 
and  hence,  there  is  no  corresponding  weight  function  that  satisfies  the  conditions  in 
Definition  8. 

Note  that  -<  in  the  above  definition  can  also  be  seen  as  a  binary  relation  as  follows. 
When  considered  between  two  sets  of  states  Si  and  S2,  f—  {(si,  S2)  G  Si  x  S2  |  3 R  C 
Si  x  S2  ■  R  is  a  strong  simulation  and  S1-RS2}.  When  considered  between  LPTSes, 
{(Li,L2)  I  L2  strongly  simulates  Lf\.  When  L\  ■<  L2,  intuitively,  L 1  is  an 
implementation  of  the  specification  L2.  The  verification  problem  we  are  interested 
in  is  to  check  whether  the  relationship  ■<  holds  between  two  LPTSes. 
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4.3.1  Properties  of  Strong  Simulation 

The  relations  □  and  A  have  some  interesting  and  useful  properties  which  we  will 
describe  here. 

Let  S  and  T  be  two  non-empty  finite  sets,  p  G  Dist(S),  v  G  Dist{T),  and  R  C 
S  x  T  such  that  p  v.  Viewing  S  and  T  as  the  two  partite  sets  of  a  bipartite 
graph  where  edges  correspond  to  R ,  we  obtain  the  following  weighted  analog  of  Hall’s 
Marriage  Theorem. 

Lemma  11  ([106]).  p  v  iff  for  every  X  C  Supp (/x) ,  p(X)  <  u(R(X)). 

Analogous  to  strong  simulation  between  non-probabilistic  labeled  transition  sys¬ 
tems  [93],  we  have  the  following  properties  of  A. 

Lemma  12.  Let  L\  and  L2  be  two  LPTSes  with  S 1  and  S2  as  the  sets  of  states, 
respectively.  Then,  AC  S\X  S2  is  the  coarsest  strong  simulation  between  L\  and  L2, 
i.e.,  A  is  a  strong  simulation  and  contains  every  strong  simulation. 

Proof.  By  Definition  9,  A  is  the  union  of  all  strong  simulations  and  hence,  contains 
every  strong  simulation.  To  show  that  A  is  a  strong  simulation,  it  suffices  to  show 
that  the  union  of  two  strong  simulations  is  a  strong  simulation.  The  latter  is  easy  to 
show  and  we  skip  the  proof.  □ 

Lemma  13  ([102]).  The  relation  A  between  LPTSes  is  a  preorder,  i.e.,  reflexive  and 
transitive. 

Proof.  Reflexivity,  i.e.,  L  A  L  for  an  arbitrary  LPTS  L,  can  be  easily  proved  by  show¬ 
ing  that  the  identity  relation  is  a  strong  simulation.  So,  we  only  consider  transitivity 
here. 
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Let  L\  =  (S'i,s?,ai,ri),  L2  =  (S2,  s°2,a2,T2) ,  and  L3  =  (S3,  s°3,a3,r3)  be  3  LPT- 
Ses  with  Li  P  L2  and  L2  P  L3.  Thus,  there  exist  strong  simulations  Ri2  C  Si  x  52 
and  R23  C  S2xS3.  We  show  that  the  relation  R  =  R23oR\2  is  a  strong  simulation  be¬ 
tween  L1  and  L3.  Let  S\Rs3  and  Si  A  p ! .  So,  there  exists  s2  G  S2  such  that  Si-Ri2s2 
and  s2R23s3.  As  RV2  is  a  strong  simulation,  there  exists  s2  A  p2  with  pi  CRl2  p2. 
Similarly,  as  R23  is  a  strong  simulation,  there  exists  S3  A  /i3  with  /i2  Lr23  /i3.  It 
suffices  to  show  that  pi  p3. 

Let  S  C  Supp(pi)  be  arbitrary.  By  Lemma  11,  we  have  p\(S)  <  P2{Rn(S))  < 
/X3 (^?23 (-R12 (A) ) )  =  p3(R(S)).  Thus,  pi  p3  and  hence,  R  is  a  strong  simulation. 

As  SiRsl ,  we  conclude  that  L\  ■<  L3.  □ 

Finally,  we  find  the  following  characterization  of  ■<  useful  in  the  algorithms  we 
will  discuss  later  on. 

Lemma  14.  Let  Li  =  (Si,Si,  T\)  be  a  tree  and  L2  =  ( S2 ,  s°,  a2,  r2)  be  an  arbitrary 
LPTS.  Let  R  C  Si  x  S2  be  such  that  for  every  s3  G  Si  and  s2  G  S2,  sii?s2  iff  the 
following  holds:  for  every  a  G  and  s3  A  pi,  there  is  a  p2  G  Dist(S'2)  with  the 
property  that  s2  A  p2  and  pi  p2.  Then,  R  =<,  i.e.,  Sii?s2  iff  s  1  s2. 

Proof.  It  suffices  to  show  that  R  and  y<C  R.  The  first  direction  easily  follows 
from  Lemma  12  as  R  is  clearly  a  strong  simulation. 

To  prove  the  other  direction,  we  first  define  the  height  of  a  state  s  G  Si  recursively 
as  follows:  the  height  of  a  leaf  state  is  defined  to  be  0  and  the  height  of  any  other 
state  is  defined  to  be  one  more  than  the  maximum  height  of  any  state  in  the  support 
of  any  outgoing  distribution  from  that  state. 
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Figure  4.6:  An  example  showing  that  Lemma  14  does  not  hold,  in  general,  if  L\  is  not  a  tree.  R  = 
{(si,  ti),  (s2,  £2)}  satisfies  the  definition  in  the  lemma,  but  R  as  -<=  {(si,  £1),  (s2,  £2),  (s2, £3)}- 


Now,  let  Si  A  s2.  We  show  that  Sii?s2  by  induction  on  the  height  of  Sx. 

For  the  base  case,  let  si  be  any  leaf  state.  As  si  has  no  outgoing  transitions, 
S1RS2  trivially  holds  by  the  definition  of  R. 

For  the  inductive  case,  let  the  height  of  Si  be  non-zero  and  let  Si  -A  fi\.  Then, 
as  ^  is  a  strong  simulation  (Lemma  12),  there  exists  /i2  with  s2  -A  /i2  such  that 
hi  h2-  Let  S  C  Supp(n  1).  Then,  by  Lemma  11,  we  have  hi(S')  <  h2^  {S)). 
As  every  state  in  Supp(n  1),  and  hence  in  S,  has  a  smaller  height  than  that  of  si, 
by  induction  hypothesis,  A  (S)  C  R(S)  and  therefore,  hi  (S')  <  h2  (R{S)).  As  S 
is  arbitrary,  we  conclude  by  Lemma  11  that  hi  h2-  By  the  definition  of  i?,  we 
obtain  that  Si_Rs2. 

Thus,  by  induction,  we  have  shown  that  R.  flj 


Note  that  the  condition  on  R  in  the  lemma  is  stronger  than  the  one  to  make  it 
a  strong  simulation  (Definition  9).  Also,  in  general,  if  Li  is  not  a  tree,  we  can  only 
conclude  that  R  C^.  See  Fig.  4.6  for  an  example  where  R  C^. 


122 


Si  R  S2 


Figure  4.7:  The  flow  network,  along  with  a  maximum  flow  from  a  to  6,  to  show  /i  C/j  v  where 
\i  €  Dist(S i),  v  €  Dist(S2),  and  the  binary  relation  R  C  Si  x  S2  are  as  in  Fig.  4.5. 

4.4  Algorithms  for  Strong  Simulation 

Strong  simulation  is  efficiently  decidable  in  polynomial  time  and  we  will  describe 
several  algorithms  for  the  problem  for  several  settings. 

Checking  C.  Let  S  and  T  be  non-empty  finite  sets  and  let  //  G  Dist(S )  and 
v  G  Dist(T).  Given  R  C  S  x  T,  one  can  check  whether  /i  n  holds  by  reducing 
it  to  a  maximum  flow  computation  problem  as  follows.  Consider  the  graph  Fll  Ti  v  = 
(S'UTU  {a,  b},  R  U  ({n}  xS)U(Tx  {6}))  denoting  a  flow  network  with  a  and  b  as  the 
source  and  the  sink  nodes.  The  edges  in  F^ji^  are  assigned  weights  according  to  the 
function  5  as  follows:  5(a,s)  =  /x(s)  for  s  G  S',  S(s,t)  =  1  for  sRt ,  and  5(t,b)  =  u(t ) 
for  t  G  T.  It  can  then  be  shown  that  F^^v  has  a  maximum  flow  of  1  from  a  to  b  iff 
/i  Cft  n  [17]  which  also  gives  an  algorithm  for  checking  Cr. 

For  example,  Fig.  4.7  shows  the  flow  network  for  distributions  /i  and  v  from 
Fig.  4.5  as  well  as  a  maximum  flow  function. 

Checking  <.  Given  two  LPTSes  L\  =  (S±,  sj,  cci,  ri)  and  L2  =  (S2,  s25  a2,  ^2),  one 
can  check  whether  Li  L2  holds  with  a  greatest  fixed  point  algorithm  that  computes 
the  coarsest  strong  simulation  between  the  LPTSes  [17].  Fig.  4.8  shows  the  pseudo- 
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code  of  the  algorithm.  The  algorithm  maintains  a  candidate  relation  R  C  S\  x  S'2, 
where  Si  and  S'2  are  the  sets  of  states  of  the  two  LPTSes,  and  iteratively  removes 
pairs  from  R  that  violate  the  condition  in  Definition  9.  The  algorithm  terminates 
when  a  fixed  point  is  reached  and  returns  the  coarsest  strong  simulation  between  Li 
and  L2.  One  can  then  check  L\  -<  L2  by  simply  examining  if  the  pair  of  start  states 
(si,  s°)  belongs  to  the  relation  returned  by  the  algorithm.  If  n  =  max(|S'i|,  | /S'2 1 )  and 
m  =  max(|ri|,  | T2 1 ) ,  this  algorithm  takes  0((mn6+m2n3)/  logn)  time  and  0(mn+n 2) 
space  in  the  worst-case  when  the  candidate  relation  R  is  implemented  as  a  queue  [  ] . 

There  exist  several  optimizations  to  this  basic  algorithm  in  the  literature  [106]. 

Reducing  ■<  to  SMT.  When  all  the  probabilities  involved  are  rational,  we  can  also 
reduce  simulation  conformance  to  satisfiability  modulo  linear  rational  arithmetic  (i.e. , 
SMT  for  the  theory  of  linear  rational  arithmetic)  as  follows,  to  take  advantage  of  the 
efficient  SMT  solvers  that  exist  today.  Given  Li  and  L2  as  above,  EncodeSim  in 
Fig.  4.9  shows  the  top-level  constraints  for  encoding  Li  -<  L2,  where  we  introduce  the 
Boolean  variables  RS1,S2  1°  denote  for  some  strong  simulation  R  and  re/WiAl2 

to  denote  fi\  p2.  Here,  AddCons  simply  adds  the  argument  to  the  pool  of 
constraints,  initialized  to  the  empty  set.  The  constraints  added  on  lines  4  and  7 
essentially  encode  the  conditions  for  L\  A  L2  from  Definition  9.  We  encode  the 
constraints  on  the  re/MljAt2  variables  on  line  6  using  EncodeDistRel  in  Fig.  4.10, 
where  we  introduce  rational  variables  'M^lit2)WiAt2,s  to  denote  the  weight  function  in 
Definition  8  and  the  variable  Sjll4l2  to  denote  the  subset  of  Supp(/i  1)  witnessing 
hi  2 R  h 2  according  to  Lemma  11.  The  constraints  added  on  lines  3-5  encode  the 
necessary  conditions  for  p\  /12  from  Definition  8  and  the  last  constraint  encodes 
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ComputeSim(Li  =  (Si,s?,ai,7i),L2  =  (S2,  s2,  a2,  r2)) 

R  4—  Si  X  S2  //  initialize  with  all  pairs  of  states 
while  true  do 

converged  V-  true 
for  (si,  s2)  £  R  do 
sim  4—  true 
for  every  s  1  A-  ji\  do 
sim  4—  false 
for  every  s2  A  g2  do 
if  Mi  E/j  M2  then 
sim  4—  true 
L  break 

if  sim  =  false  then 

L  break 

if  sim  =  false  then 

//  Si  ^  s2 
R  R  \  {(si,  s2)} 

converged  4—  false 
L  break 

if  converged  then 

//  fixed-point  reached 
L  return  R 


Figure  4.8:  Greatest  fixed-point  algorithm  for  computing  the  coarsest  strong  simulation  relation 
between  two  LPTSes  L\  and  L2. 


the  (contrapositive  of  the)  sufficient  condition  from  Lemma  11.  The  set  ,/t2  and 
its  image  under  R  can,  in  turn,  be  encoded  using  auxiliary  Boolean  variables  for  the 
states  in  Si  and  S2,  but  we  leave  the  details  to  the  reader. 

The  following  is  immediate. 

Lemma  15.  Li  ^  L2  iff  the  constraints  resulting  from  EncodeSim(Li, L2)  are 
satisfiable. 

Checking  conformance  for  trees.  We  also  consider  a  specialization  of  the  greatest 
fixed-point  algorithm  when  Li  is  a  tree,  which  is  used  during  abstraction  refinement 
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EncodeSim(Li  =  (Si,  sj,  ai,Ti),L2  =  (S2,s%,a2,r2)) 

introduce  Boolean  variables  Rsi,s2  to  denote  (si,S2)  G  R  C  Si  X  S2 
introduce  Boolean  variables  rel^,  )/i2  to  denote  //i  C/j  ^2 
for  euery  (si,  S2)  £  S\  x  S2  do 

L  AddCons (R,im  =*■  A{(0iM)|,lAM)VtB|„Am)  "=!«,«) 

for  euery  (m, /x2)  £  Dist(5'i)  x  Dist(5,2)  do 

//  only  the  distributions  appearing  in  the  transitions 

EncodeDistRel(/xi,  n2,  R,  relln :IJi2 ) 


7 


Add  Cons  (i?so  so ) 


Figure  4.9:  SMT  encoding  for  L\  ■<  L2- 
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EncodeDistRel(^i  g  Dist(Si),fi2  £  Dist(S2),R,b) 
introduce  rational  variables  Wtltt2  for  (ti,t2)  £  Si  X  S2 
let  S  denote  a  variable  denoting  a  subset  of  Si 
AddCons (b  =►  A<ieSl  Mi(*i)  =  Et2Gs2  «W2) 

AddCons(6  =>  At2es2  ^(t2)  =  £tieSl  "'ti.ta) 

AddCons(6  =►  /\t1es1,t2es2  wti,t2  >  0  =* 

AddCons(-'6  /ii(5)  >  fi2(R(S))) 

//  S  and  R(S)  can  further  be  encoded  with  Boolean  variables 


Figure  4.10:  SMT  encoding  for  pj  p2-  Here,  b  is  a  Boolean  variable  denoting  the  truth  value  of 
Mi  E.R  M 2- 


(Sections  6.2  and  6.3).  Fig.  4.11  shows  the  pseudo-code  of  the  algorithm  which  is 
based  on  a  bottom-up  traversal  of  L\.  It  maintains  a  candidate  relation  R ,  initialized 
to  Si  x  S2.  For  every  non-leaf  state  si  G  Si  in  a  bottom-up  traversal  of  Li,  the 
algorithm  iteratively  checks  if  a  transition  si  -A  //  L  can  be  simulated  by  L2  and  for 
every  s2  G  S2  that  does  not  have  a  simulating  transition  on  a,  removes  the  pair 
(si,  S2)  from  R.  Correctness  can  be  shown  by  induction  on  the  height  of  a  state  in 
Si  and  we  leave  the  details  to  the  reader. 
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ComputeSimTree(Li  =  (Si,  s°,  cu,  n),  L2  =  (S2,  s2,  a2,  t2)) 

J?.  <-  5i  x  ^2  //  initialize  with  all  pairs  of  states 
for  every  non-leaf  sj  £  Si  in  a  bottom-up  traversal  of  Li  do 
for  every  si  A  Mi  do 

for  every  s2  G  R(s\)  do 
sim  4—  false 
for  every  s2  A  g2  do 
if  Mi  Er  M2  then 
sim  4—  true 
L  break 

if  sim  =  false  then 

|_  R  4—  R  \  {(si,  s2)} 

_  return  R 

Figure  4.11:  Specialized  fixed-point  algorithm  for  computing  the  coarsest  strong  simulation  between 
L\  and  L2  when  L\  is  a  tree. 

4.5  Counterexamples  to  Strong  Simulation 


We  have  seen  several  efficient  algorithms  in  the  previous  section  for  deciding  strong 
simulation  between  two  LPTSes.  However,  our  techniques  for  automatic  composi¬ 
tional  reasoning  are  iterative  and  in  order  to  recover  from  the  cases  where  strong 
simulation  fails  to  hold,  we  also  need  to  characterize  the  notion  of  a  counterexample 
to  the  conformance  relation.  We  first  define  a  counterexample  using  a  language- 
theoretic  formulation  of  strong  simulation  and  then  characterize  counterexamples  as 
stochastic  trees. 

Definition  10  (Language  of  an  LPTS).  Given  an  LPTS  L,  we  define  its  language, 
denoted  C(L),  as  the  set  of  all  LPTSes  simulated  by  it,  i.e.,  {L'  \  L'  is  an  LPTS  and 
L'  P  L}. 

We  immediately  have  the  following  result. 

Lemma  16.  For  LPTSes  L\  and  L2,  L\  L2  iff  C(Li)  C  C(L2). 
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Proof.  We  know  from  Lemma  13  that  ^  is  transitive  and  reflexive.  In  the  above 
statement,  necessity  follows  from  the  transitivity  of  ■<  and  sufficiency  follows  from 
the  reflexivity  of  which  implies  Li  G  C(Lf).  □ 

So,  a  counterexample  to  strong  simulation  can  be  defined  as  follows. 

Definition  11  (Counterexample).  Given  LPTSes  L\  and  L2  with  L\  L2,  a  coun¬ 
terexample  is  an  LPTS  C  such  that  C  G  £(Li)  \  C{L2),  i.e.  C  ■<  L\  but  C  ^  L2. 

Now,  Li  itself  is  a  trivial  choice  for  C  but  it  does  not  give  any  more  informa¬ 
tion  than  what  we  had  before  checking  the  simulation  conformance.  So,  we  are 
interested  in  counterexamples  with  simpler  structure  which  retain  the  relevant  infor¬ 
mation  to  witness  the  failure  of  the  conformance  relationship.  When  the  probability 
distributions  are  all  restricted  to  Dirac  distributions,  i.e.,  when  we  consider  LTSes, 
a  tree-shaped  LTS  is  known  to  be  sufficient  as  a  counterexample  [32].  Based  on  a 
similar  intuition,  we  show  that  a  stochastic  tree  is  sufficient  as  a  counterexample  for 
simulation  conformance  between  arbitrary  LPTSes. 

Theorem  10.  Given  LPTSes  L\  =  (Si,  s?,  ot\,  rf)  and  L2  =  (S'2,  a2,  ^2)  with 

L\  ii  L2,  there  exists  a  tree  counterexample. 

Proof.  For  i  G  {1,2},  let  (Lj,s)  denote  the  LPTS  which  is  the  same  as  Li  except 
that  the  start  state  is  s  instead  of  s°,  i.e.,  (Li,  s)  =  (Si,  s,  rf). 

Consider  the  greatest  fixed  point  algorithm  COMPUTESlM  in  Fig.  4.8  for  com¬ 
puting  the  coarsest  strong  simulation  between  two  LPTSes.  Let  R  C  S\  x  S'2  be  the 
relation  maintained  by  ComputeSim(Li,  L2).  We  show  that  whenever  a  pair  (si,  s 2) 
is  removed  from  R ,  there  is  a  tree  T]2  which  is  a  counterexample  to  (Li,s\)  (L2,  S2). 
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As  Li  L2,  the  pair  (sj,  s2)  is  eventually  removed  from  R  and  it  will  follow  that 
a  tree  counterexample  to  Li  ■<  L2  exists.  We  proceed  by  strong  induction  on  the 
number  of  iterations  of  the  outermost  while  loop  of  COMPUTESlM. 

In  the  base  case,  R  =  S±  x  S2  and  p\  p2  holds  for  every  // 1  E  Dist{S i) 
and  [1 2  E  Dist(S2).  So,  when  (si,s2)  is  removed  from  i?,  it  must  be  the  case  that 
there  is  a  transition  si  A  p\  such  that  no  transition  exists  from  s2  on  action  a. 
Now,  let  Ti2  be  the  tree  representing  the  transition  si  A  p\  by  creating  a  new 
state  ti  for  Si  and  a  new  state  ts  for  every  s  E  Supp(pi),  i.e. ,  T12  =  ({A}  U  {ts  \ 
s  E  Supp(ni)},ti,{a},{(t1,a, p\)}),  where  p\(ti)  =  0  and  p\(ts)  =  pi(s)  for  s  E 
Supp(n i).  It  can  be  easily  shown  that  TV2  is  a  counterexample  to  (Li,s i)  A  (L2,s 2), 
i.e.,  T12  A  (Li,  si)  but  T12  (L2,s2). 

We  will  now  consider  the  inductive  case  and  let  a  new  pair  (si,s2)  be  removed 
from  the  current  R.  So,  there  is  a  transition  Si  A  pi  which  is  simulated  by  no 
transition  on  a  from  s2.  Let  A  =  {u  E  Dist(S2 )  |  s2  A  z/}.  So,  we  know  that 
Li  2 R  v  f°r  every  v  E  A.  The  case  of  A  =  0  is  the  same  as  the  base  case  above.  So, 
we  assume  that  A  is  non-empty  below. 

Let  v  E  A.  Because  ji\  [ZA  p,  there  exists  a  set  S\  C  Supp(pi)  such  that 
Pi (SA  >  u(R(S'0)  by  Lemma  11.  Let  =  Supp(u )  \  R(S'().  Now,  for  every  pair 
(u,v)  E  S"  x  it  follows  that  (u,  v)  R  and  by  inductive  hypothesis,  there  exists 
a  tree  counterexample  TU}V  for  (Li,u)  A  (L2,u). 

We  build  a  tree  T\2  as  follows.  We  describe  the  construction  at  a  high  level  and 
leave  the  details  to  the  reader.  As  in  the  base  case,  we  start  with  representing  the 
transition  si  A  p\  as  a  tree,  say  To,  by  creating  a  new  state  t\  for  si  and  a  new  state 
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CexDistRel(/x  g  Dist(Si),  v  G  Dist(S2),R  C  Si  x  S2) 

/  :=  a  maximum-flow  function  for  the  flow  network  (see  Section  4.4) 

find  S!  G  Si  with  /x(si)  >  J2S2eS2  /(s  1^2) 

Si"  =  {*1} 

while  /x(Sf)  <  do 

|_  S\  :=  {si  G  Si  |  exists  s2  G  R(S 1)  with  f(s±,  s2)  >  0} 

_  return  S \ 

Figure  4.12:  Finding  S”  C  S\  such  that  p{S")  >  v(R{Si))i  given  /.x  z'- 

for  every  s  G  Suppfa  1),  i.e.,  T0  =  ({ti}  U  {ts  |  s  G  £1,  {a},  {(£1,  a, 

where  =  0  and  n\(ts)  =  /ii(s)  for  s  G  Supp(ni).  Then,  for  every  (u,v)  G 

U^ga  (^1  x  S%),  we  attach  the  tree  Tuv  to  the  state  tu  in  T0,  i.e.,  we  merge  the 
start  state  of  TUjV  with  tu.  We  claim  that  T12  so  obtained  is  a  counterexample  to 
(Li,si)  (L2,  s2)  which  we  show  below. 

First  of  all,  it  can  be  easily  shown  that  T12  A  (Ll5  si)  as  TV2  is  essentially  a  finite 
unwinding  of  (Li,si).  So,  we  will  only  show  that  7\2  ^  (L2,s2).  Let  u  G  A  and 
let  i?i2  be  the  coarsest  strong  simulation  between  Ti2  and  (L2,  s2).  Consider  the  set 
S  =  {tu  |  u  G  S']y}.  Now,  by  construction,  we  know  that  for  every  (u,v)  G  x  S%, 
( Ti2,tu )  is  a  counterexample  to  (Li,w)  A  (L2,v)  and  in  particular,  we  know  that 
(Ti2,tu)  (L2,u).  This  implies  that  (tu,v)  R±2  for  every  such  (u,v).  In  other 
words,  Ri2(S)  C  Therefore,  yu* (S')  =  Ri(S”)  >  i/(R(Si))  >  is(Ri2(S))  and 

by  Lemma  11,  we  conclude  that  /j\  % r12  is.  As  v  G  A  is  arbitrary,  this  implies  that 
(ti,s2)  0  R\2  and  hence,  T12  (L2,s2).  □ 

See  Fig.  4.13  for  an  illustration  of  a  counterexample.  Now,  in  order  to  obtain  an 
algorithm  for  computing  a  tree  counterexample  from  the  proof  of  Theorem  10  above, 
it  remains  to  show  how  to  compute  a  subset  S"  C  Supp(p)  that  acts  as  a  witness  for 
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p  v  for  distributions  /i  G  Dist(Si),  v  G  Dist(S2)  and  R  C  S±  x  S2 .  See  Fig.  4.12 
for  an  algorithm  to  compute  such  a  witness  subset  which  is  analogous  to  finding  a 
subset  failing  Hall’s  condition  in  Graph  Theory  and  can  easily  be  proved  correct. 

We  have  the  following  complexity  result  for  computing  counterexamples. 
Theorem  11.  Given  LPTSes  Li  and  L2,  deciding  Li  ■<  L2  and  obtaining  a  tree  coun¬ 
terexample  when  conformance  fails  to  hold  takes  0(mn6+m2?i3)  time  and  0(mn+n2) 
space  where  n  =  maxdS^J,  |Sl2|)  and  m  =  max(|ri|,  |t2|). 

Proof.  It  can  be  easily  be  seen  that  Algorithm  4.12  takes  0{n3)  time  and  O(n) 
space  which  increases  the  complexity  of  checking  p\  Cr  p2  to  0(n3)  time  and  0(n2) 
space  (see  Section  4.4  for  an  algorithm  to  decide  Cr).  The  rest  of  the  argument 
is  similar  to  that  of  the  greatest  fixed-point  algorithm  for  computing  the  coarsest 
strong  simulation  [17].  □ 

Note  that  the  tree  counterexample  C  to  L\  ■<  L2  constructed  by  the  algorithm 
in  the  proof  of  Theorem  10  is  essentially  a  finite  tree  execution  of  L\.  That  is,  one 
can  readily  obtain  a  total  mapping  M  :  Sc  — >  Si,  where  Sc  is  the  set  of  states 
of  C ,  with  the  following  property:  for  every  transition  c  A  /ic  of  C,  there  exists 
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M(c)  A-  /i|  such  that  M  is  an  injection  when  restricted  to  Supp(/ac )  and  for  every 
d  G  Supp(pc),  /ic(c')  =  /Ui(M(c/)).  One  can  easily  show  that  M  is  also  a  strong 
simulation.  We  call  such  a  mapping  an  execution  mapping  from  C  to  L^.  Fig.  4.13 
shows  an  execution  mapping  in  brackets  beside  the  states  of  C.  Note  also  that, 
in  the  inductive  case  of  the  proof  of  the  above  theorem,  attaching  trees  to  a  state 
s  of  C,  using  the  inductive  hypothesis,  can  result  in  multiple  copies  of  the  same 
transition  of  L\  outgoing  from  s.  This  can,  however,  be  avoided  with  additional 
bookkeeping.  This  gives  us  the  following  corollary,  which  essentially  says  that  we 
can  always  obtain  a  counterexample  without  non-determinism  as  long  as  L\  does  not 
have  non-determinism. 

Corollary  2.  If  Li  is  reactive  and  L\  -ff  L2,  there  exists  a  reactive  counterexample. 

While  Theorem  10  shows  that  a  tree  counterexample  always  exists  when  simula¬ 
tion  conformance  fails  to  hold,  it  is  not  immediately  clear  whether  the  tree  structure 
(multiple  outgoing  transitions  from  a  state)  is  really  necessary.  The  following  lemma 
shows  that  it  is  indeed  necessary,  in  general. 

Lemma  IT.  There  exist  reactive  LPTSes  R\  and  R2  such  that  Ri  ^  R2  and  no 
counterexample  is  fully-probabilistic. 

Proof.  Consider  the  two  reactive  LPTSes  R\  and  R2  in  Fig.  4.14.  The  states,  actions, 
and  distributions  of  the  LPTSes  are  labeled  as  in  the  figure.  It  is  easy  to  see  that 
t  11  ii  t 21,  r n  y<  r 23,  and  rn  ^  r22.  It  follows  that  /i10({ru})  =  \  >  /a20 (P  ({rn})  = 
P2o({r22})  =  |  and  hence,  /i  1 0  [2^  M20  (Lemma  11).  Therefore,  r10  r20  and  hence, 

R 1  -^2- 

Let  us  assume,  for  the  sake  of  contradiction,  that  there  is  a  fully-probabilistic 
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Figure  4.14:  An  example  where  there  is  no  fully-probabilistic  counterexample. 


counterexample  C  and  let  its  initial  state  be  Co-  Thus,  C  ■<  R\  but  C  ^  U2.  By 
Definition  9  there  exists  a  strong  simulation  U  such  that  (co,no)  G  U .  Given  that  C 
is  a  counterexample  and  is  full-probabilistic,  Co  has  exactly  one  outgoing  transition, 
say  Co  A  po-  Note  that  this  transition  must  be  labeled  by  x  for  it  to  be  simulated 
by  R\.  Let  c  1  be  an  arbitrary  state  in  Supp(p 0).  If  Ci  has  any  outgoing  transitions, 
it  implies  that  (ci,ri2)  U.  Moreover,  in  that  case,  as  po  Q u  P10,  U  must  include 
(ci,m)  and  hence,  the  (only)  transition  from  Ci  must  be  labeled  either  y  or  z.  So, 
let  Ci  A  p\.  Now,  the  only  transition  on  y  from  rn  leads  to  the  distribution  p±w 
and  in  order  to  have  p\  □ u  /i110,  every  state  in  Supp(pi)  must  be  related  to  r13  by  U 
and  hence,  have  no  outgoing  transitions.  One  can  reach  a  similar  conclusion  if  the 
outgoing  transition  from  Ci  is  on  the  action  0  instead. 

To  summarize  our  inferences  about  C,  it  must  be  a  tree  with  exactly  one  transition 
from  the  initial  state  Co,  say  Co  A  po,  such  that  for  every  state  in  Supply 0),  there 
can  at  most  one  transition,  which  can  only  be  labeled  either  y  or  z  and  there  are  no 
other  transitions.  Let  Sy  and  Sz  be  the  sets  of  states  in  Supp(p 0)  with  an  outgoing 
transition  labeled  by  y  and  z,  respectively.  As  po  Qu  hio,  we  have  po(Sy  U  Sz)  < 
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A ho(U ( Sy  U  Sz))  =  /iio({rn})  =  i.e.,  fjto(Sy  U  Sz)  <  §. 

Let  V  be  the  smallest  binary  relation  between  the  states  of  C  and  R2  that  satisfies 
the  following  conditions.  V  relates  the  initial  states  Cq  and  r2o •  Let  c  be  an  arbitrary 
state  of  C  other  than  the  initial  state.  If  c  has  no  outgoing  transitions,  V  relates  c 
to  every  state  of  R2.  If  c  has  its  transition  labeled  by  y,  V  relates  it  to  r2 1  and  r22. 
If  c  its  transition  labeled  by  z,  V  relates  it  to  r22  and  r2 3. 

We  show  that  y0  (Ty  P2o-  Let  X  C  Supply 0)  be  arbitrary.  If  X  includes  a 
state  with  no  transitions,  V(X)  =  S2,  the  set  of  all  states  of  R2  and  hence  yo(X)  < 
y20(V(X))  =  1.  Otherwise,  X  only  has  states  with  transitions  labeled  by  either 
y  or  z,  i.e.,  X  C  Sy  U  Sz,  and  by  the  observation  made  in  the  above  paragraph, 
Ho{X)  <  \  whereas  y2o(V(X))  >  |.  Thus,  yo(X)  <  y2o(V(X)).  This  implies  that 
C  <  R2  which  contradicts  the  assumption  that  C  is  a  counterexample.  □ 

Thus,  even  if  Li  does  not  have  non-determinism,  i.e.,  Li  is  reactive,  the  above 
theorem  shows  that  a  counterexample  must  have  the  tree  structure  (multiple  transi¬ 
tions  outgoing  from  a  state),  in  general.  This  is  surprising,  as  the  non-probabilistic 
counterpart  of  a  fully-probabilistic  LPTS  is  a  trace  of  actions  and  it  is  known  that 
trace  inclusion  coincides  with  simulation  conformance  between  reactive  (i.e.,  deter¬ 
ministic)  LTSes.  On  a  related  note,  if  L 1  is  allowed  to  have  non-determinism,  one 
may  ask  if  a  reactive  LPTS  suffices  as  a  counterexample  to  L\  -<  L2.  That  is  not  the 
case  either,  as  the  following  lemma  shows. 

Lemma  18.  There  exist  an  LPTS  L  and  a  reactive  LPTS  R  such  that  L  7^  R  and 
no  counterexample  is  reactive. 
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Figure  4.15:  There  is  no  reactive  counterexample  to  L  d  R. 


Proof.  Consider  the  LPTS  L  and  the  reactive  LPTS  R  in  Fig.  4.15.  The  states, 
actions,  and  distributions  are  labeled  as  in  the  hgure.  It  is  easy  to  see  that  yno  2d 
/r23  and  /ini  2d  P21  whereas  yno  !2d  P21,  P22  and  /im  ^22,^23-  It  follows  that 
Pio({hi})  =  l>  P2o(^  {{hi}))  =  h2o({^22})  =  |  and  hence,  /ii0  2d  P20-  Therefore, 
/10  2  r20  and  hence,  L  2  R- 

Let  us  assume,  for  the  sake  of  contradiction,  that  there  is  a  reactive  counterex¬ 
ample  C  and  let  its  initial  state  be  Cq.  Similar  to  the  arguments  made  in  the  proof 
of  Lemma  17,  one  can  show  that  C  must  be  a  tree  with  exactly  one  transition  from 
the  initial  state,  say  Co  A-  yo,  such  that  for  every  state  in  Supp{y 0),  there  can  be 
at  most  one  transition,  which  can  only  be  labeled  y  (because  hi  has  transitions 
only  on  y).  Let  Ci  G  Supply 0)  be  such  that  Ci  -h  y\.  It  is  also  the  case  that  all 
transitions  (if  any)  from  a  state  in  Supply  1)  must  be  labeled  by  the  same  action, 
which  can  only  be  either  2  or  w.  Let  Sy  denote  the  subset  of  states  in  Supp{y$)  with 
outgoing  transitions  (which  can  only  be  labeled  y).  Then,  one  can  also  show  that 
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H0(Sy)  <  /^lod^ll})  =  \' 

Let  V  be  the  smallest  binary  relation  between  the  states  of  C  and  R  that  satisfies 
the  following  conditions.  V  relates  the  initial  states  Co  and  r2o ■  Let  c  be  an  arbitrary 
state  of  C  other  than  the  initial  state.  If  c  has  no  outgoing  transitions,  V  relates  c 
to  every  state  of  R.  If  c  has  a  transition  labeled  by  either  z  or  w,  V  relates  it  to  all 
states  of  R  that  have  transitions  labeled  z  or  w,  respectively.  On  the  other  hand,  if  c 
has  a  transition  labeled  by  y,  say  cA/i,  depending  on  whether  the  states  in  Supply) 
have  transitions  on  z,  w,  or  none,  V  relates  c  to  r2\  and  r22,  or  r22  and  r23,  or  to 
all  three  of  721,  r22  and  r2^.  One  can  show  that  V  is  a  strong  simulation  implying 
C  8  R.  This  contradicts  the  assumption  that  C  is  a  counterexample.  i.j3 


4.6  Composition  of  LPTSes 

Parallel  composition  between  LPTSes  is  defined  in  the  usual  way  by  means  of  syn¬ 
chronization  on  common  actions.  As  the  transitions  in  LPTSes  are  probabilistic,  we 
need  the  notion  of  a  product  of  two  distributions  that  multiplies  the  probabilities 
point-wise,  defined  as  follows.  Given  two  finite  sets  S  and  T,  and  two  distributions 
y  G  Dist(S)  and  v  G  Dist(T),  the  product  of  y  and  u,  denoted  is  a  distribution 

over  S  x  T  such  that  (y  (8)  zz)(s,  t)  =  y(s)  x  v(t)  for  every  s  G  S  and  t  G  T. 

Definition  12  (Composition  |  ]).  Let  Li  =  (Si,  s°,  an,  T\)  and  L2  =  (S2,  s2,a2)T2) 

be  two  LPTSes.  The  parallel  composition  of  L\  and  L2,  denoted  L\  ||  L2,  is  defined 
as  the  LPTS  ( S,s°,a,r ),  where  S  =  S\  x  S2,  s°  =  a  =  U  a2,  and 

((•Si,  s2),a,  y)  G  t  iff  one  of  the  following  holds: 
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Figure  4.16:  Three  LPTSes  such  that  L  is  the  parallel  composition  L\  ||  L2. 


1.  p  =  p\  <g)  p2  for  some  transitions  si  A  p\  and  s2  A  p2  (a  is  common ) 

2.  a  ^  a 2  and  p  —  pi®i  dS2  for  some  transition  si  A  /xi  (a  zs  /oca/  fo  L\) 

3.  a  a.  1  aru/  p  =  <5Sl  <g)  p2  for  some  transition  s2  A  p2  (a  /oca/  to  L2) 

For  example,  in  Fig.  4.16,  L  =  ||  L2.  We  have  the  following  useful  property. 

Lemma  19  ([102]).  ■S  is  compositional.  That  is,  let  Li  =  (S±,  s®,  cti,  Ti)  and  L2  = 
(S2, s2,  ol2 ,  t2)  6e  /too  LPTSes  such  that  Li  P  L2  and  a2  C  a1.  Then,  for  every  LPTS 
L,  L\  ||  L  P  L2  ||  L. 


Proof.  Let  L1;  L2  and  L  be  as  in  the  statement  and  let  L  =  {SL,sf,aL,ri).  So, 
by  Definition  9,  there  exists  a  strong  simulation  R\2  C  Si  x  S2  such  that  (s°,s2)  G 
Ri2.  Consider  the  relation  R  =  {((si,  s),  (s2,  s))  |  s\R\2s2  and  s  G  Sh}-  Clearly, 
(s®,  s°l)R(s2,  s°l ).  It  suffices  to  show  that  R  is  a  strong  simulation  between  Li  ||  L 
and  L2  ||  L. 

Let  (si,  s)R(s2 ,  s )  and  (si,  s )  A  pa  be  arbitrary  for  some  action  a  G  a  1  U  ct^.  By 
definition  of  R,  S\Ri2s2.  By  Definition  12,  there  are  three  cases  to  analyze. 

si  A  /ii,  s  A  p  and  pa  =  pi  ®  p  '■  As  siR\2s2  and  R\2  is  a  strong  simulation,  there 
exists  a  transition  s2  A  p2  with  pi  Gr12  p2.  By  Dehnition  12,  (s2,s)  A  p'a 
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where  \j!a  =  p2  <8>  /i-  Let  X  C  Supp(pa).  For  each  s  G  Sd,  dehne  X{  =  {£x 
(fi,s)  G  X}.  We  have 

MV)  =  E  MV!  x  W) 

s£Sl 

sgSl 

<  Y  mmmd)  ■  M) 

seSL 

=  E  MM (V,1)  X  {s}) 

S(zSl 

=  ]T  MMV,1  x  {a})) 

s£Sl 

=  M  U  fl(V,‘  x  {s})) 

sGSl 

=  M*(  U  A',1  x  {a})) 

s£Sl 

=  M-R(V)). 

As  X  is  arbitrary,  it  follows  from  Lemma  11  that  pa 

a  0  ai,  sA/i  and  pa  =  &si®  V  '■  As  a2  «i,  a  0  a2  and  by  Dehnition  12,  (s2,  s)  — > 
[j!a  with  n'a  =  (5S2  <g)  fi.  Now,  let  A"  C  Supp(fia )  and  let  X"2  =  {t  \  (si,t)  G  X}. 
We  have  /ia(X)  =  /i(X2)  =  /<({s2}  x  X2)  =  n'a(R(X))  and  hence,  pta  C R  p'a. 

s\  ii\,  a  £  oil  and  pa  =  /m  <g)  5S  :  As  sii?i2s2  and  i?i2  is  a  strong  simulation, 
there  exists  s2  A  /i2  with  // j  C^12  /i2.  Now,  let  X  C  Supp(pa )  and  let  Ad  = 
{*i  I  (t!,s)  G  X}.  We  have  /ia(X)  =  ^(Xj)  <  =  /4(i2(X))  and 


{definition  of  yUa} 
{i?i2  is  a  strong  simulation} 
{dehnition  of 
{dehnition  of  R} 
{the  sets  R( A"}  x  {s})  are  disjoint} 
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hence,  /ia  C R  p!a. 

Hence,  R  is  a  strong  simulation.  Therefore,  L\  ||  L  -<  L2  ||  L.  □ 

Finally,  we  show  the  soundness  and  completeness  of  the  assume-guarantee  infer¬ 
ence  rule  ASym  mentioned  in  Chapter  1,  reproduced  below.  Here,  L 1,  L2,  A,  and  P 
are  all  LPTSes. 


1  :  Li  \\  A  P  P  2  \L2<A 
- (ASym) 

L\  ||  L2  A  P 

The  rule  is  sound  if  the  conclusion  holds  whenever  the  premises  hold  for  some 
assumption  LPTS  A,  and  the  rule  is  complete  if  there  is  an  assumption  A  satisfying 
the  premises  whenever  the  conclusion  holds. 

Theorem  12.  If  a  a  C  a2?  the  rule  ASym  is  sound  and  complete. 

Proof.  Soundness  follows  from  Lemma  19.  Completeness  follows  trivially  by  replac¬ 
ing  A  with  L2-  □ 
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Chapter  5 


Active  Learning  for 
Simulation  Conformance 


5.1  Introduction 


We  have  seen  in  Chapter  4  that  strong  simulation  conformance  between  two  Labeled 
Probabilistic  Transition  Systems  (LPTSes)  is  decidable  in  polynomial  time.  How¬ 
ever,  as  mentioned  in  Chapter  1,  when  an  LPTS  L  is  the  parallel  composition  of 
multiple  components,  we  encounter  the  state-space  explosion  problem  for  checking 
conformance  with  a  specification  LPTS  P.  To  address  this  problem,  we  follow  the 
assume-guarantee  paradigm  [98 1  for  compositional  reasoning.  In  particular,  we  focus 
on  the  following  assume-guarantee  inference  rule,  which  we  have  shown  to  be  sound 
and  complete  (see  Chapter  4): 


141 


1  :  Lx  ||  A  A  P  2  \L2<A 
-  (ASym) 

L\  ||  L2  ^  P 

In  other  words,  in  order  to  show  that  a  probabilistic  system  composed  of  two  par¬ 
allel  components  L\  and  L2  conforms  to  a  specification  P,  it  suffices  to  come  up  with 
a  (preferably  small)  assumption  A  about  L2  which  can  be  used  in  its  place  to  show 
the  conformance  together  with  L\.  In  this  chapter,  we  study  iterative  algorithms  for 
learning  a  small  assumption  A,  given  L1?  L2,  and  P,  from  counterexamples  to  the 
premises. 

In  the  context  of  non-probabilistic  systems,  several  algorithms  exist  for  compo¬ 
sitional  verification  that  are  based  on  learning  the  intermediate  assumptions  from 
samples  generated  dynamically.  In  particular,  algorithms  for  compositional  verifica¬ 
tion  of  trace  inclusion  [33,  99]  and  simulation  conformance  [31]  have  been  studied 
that  are  based  on  learning  from  traces  [14,  96]  and  trees  [31],  respectively.  These 
algorithms  are  essentially  adaptations  of  active  learning  [14]  algorithms  for  inferring 
an  unknown  target  system  from  samples,  to  the  compositional  setting.  An  active 
learning  framework  typically  has  two  entities  -  a  learner  which  tries  to  learn  the 
unknown  target  and  a  teacher  which  guides  the  learner  by  giving  new  information  in 
terms  of  samples.  The  teacher,  typically,  can  answer  two  types  of  queries  -  member¬ 
ship  (of  a  sample  in  the  unknown  target)  and  equivalence  (between  the  conjectured 
model  and  the  unknown  target)  [14].  The  learner  terminates  when  an  equivalence 
query  is  answered  positively  by  the  teacher.  In  the  context  of  assume-guarantee  style 
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compositional  reasoning,  the  unknown  target  corresponds  to  a  sufficient  assumption 
in  the  inference  rule  and  the  teacher  answers  the  queries  by  checking  the  premises  of 
the  rule  [99] . 

However,  compositional  reasoning  for  probabilistic  systems  has  not  been  studied 
well  in  the  literature.  In  particular,  no  algorithms  are  known  (based  on  learning 
or  otherwise)  for  compositional  verification  of  simulation  conformance.  For  non- 
probabilistic  systems,  simulation  conformance  between  two  labeled  transition  sys¬ 
tems  (LTSes)  reduces  to  tree  language  inclusion  and  there  exists  an  adaptation  of 
an  active  learning  algorithm  for  deterministic  tree  automata  to  the  compositional 
setting  [31].  Now,  in  the  probabilistic  setting,  we  have  seen  in  Chapter  4  that  a 
counterexample  to  strong  simulation  between  LPTSes  is  a  stochastic  tree.  So,  we 
can  similarly  define  a  stochastic  tree  language  such  that  strong  simulation  reduces  to 
inclusion  between  stochastic  tree  languages.  However,  while  there  exist  techniques 
for  learning  from  samples  consisting  of  (non-stochastic)  trees  with  information  re¬ 
garding  the  probability  of  acceptance  [28],  we  are  not  aware  of  any  prior  algorithms 
for  learning  from  stochastic  trees.  Moreover,  we  are  also  not  aware  of  a  probabilistic 
variant  of  a  tree  automaton  to  recognize  stochastic  tree  languages.  This  motivated 
us  to  consider  learning  an  LPTS  directly  from  stochastic  tree  samples,  as  opposed  to 
inventing  stochastic  tree  automata  and  casting  the  verification  problem  in  automata- 
theoretic  terms. 

In  our  context  of  active  learning,  an  equivalence  query  corresponds  to  asking 
whether  a  conjecture  C  is  strong  simulation  equivalent  to  T,  i.e.,  whether  C  -<  T 
and  T  <  C .  So,  when  the  equivalence  check  fails  for  a  conjecture  C,  a  counterexample 
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LearnLPTS() 

V  :=  0,  Af  :=  0  //  initialize  positive  and  negative  tree  samples 
while  true  do 

L  :=  FlNDCONSISTENT(73,.A/’)  //  see  Section  5.2 
(res,  cex)  :=  CheckConjecture(L)  //  ask  teacher 
if  res  is  j/es  then 

II  L  is  equivalent  to  target 
L  return  L 

else  if  res  is  positive  then 

//  L  does  not  simulate  target,  witnessed  by  cex 

V  :=  V  U  {cex} 

else  if  res  is  negative  then 

II  target  does  not  simulate  L,  witnessed  by  cex 

Af  :  =  WU  {cex} 


Figure  5.1:  Active  learning  loop  for  inferring  an  LPTS  using  only  equivalence  queries. 


can  be  of  two  kinds.  If  T  ^  C,  then  a  counterexample  tree  t  satisfies  t  <  T  but 
t  ^  C  and  we  call  it  a  positive  sample.  On  the  other  hand,  if  C  ^  T,  then  a 
counterexample  tree  t  satisfies  t  ■<  C  but  t  ^  T  and  we  call  it  a  negative  sample. 
Now,  a  membership  query  would  correspond  to  asking  whether  a  stochastic  tree  t 
is  simulated  by  the  unknown  target  LPTS  T,  i.e. ,  whether  t  y<  T.  However,  we 
observe  that  such  a  membership  query  is  challenging  to  create  as  the  learner  would 
need  to  guess  not  only  the  tree  structure  but  also  the  transition  probabilities.  For 
this  reason,  we  restrict  the  learning  framework  such  that  a  teacher  can  only  answer 
equivalence  queries. 


Fig.  5.1  shows  the  pseudo-code  of  our  active  learning  framework  LearnLPTS. 
The  learner  maintains  a  set  of  positive  and  negative  tree  samples.  In  each  iteration, 
it  infers  an  LPTS  L  consistent  with  all  the  samples,  i.e.,  L  simulates  all  the  positive 
samples  and  none  of  the  negative  samples,  and  conjectures  L  to  the  teacher.  If  the 
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teacher  finds  L  to  be  equivalent  to  the  target,  it  returns  yes ,  and  otherwise,  it  returns 
a  new  positive  or  a  new  negative  sample. 

We  first  describe  algorithms  for  FlNDCONSISTENT,  i.e. ,  for  inferring  an  LPTS 
consistent  with  a  given  set  of  positive  and  negative  samples,  in  Section  5.2.  Given 
the  ultimate  objective  of  inferring  a  small  assumption  in  the  rule  ASym,  our  main 
interest  is  in  learning  consistent  LPTSes  of  small  size.  To  this  end,  our  algorithms 
employ  two  different  ways  of  partitioning  the  state-space  of  the  counterexamples. 
We  then  describe  the  convergence  guarantees  of  LearnLPTS  in  Section  5.3.  In 
particular,  we  show  that  there  is  no  converging  learning  algorithm  in  the  presence 
of  an  adversarial  teacher,  but  there  exists  a  converging  algorithm  under  a  natural 
assumption  on  the  teacher.  We  also  discuss  how  convergence  is  affected  when  the 
consistent  LPTS  in  each  iteration  is  required  to  have  the  minimal  number  of  states. 
Finally,  in  Section  5.4,  we  describe  how  active  learning  is  adapted  for  compositional 
reasoning  and  discuss  the  complexity  guarantees. 


5.2  Learning  a  Consistent  LPTS 

Assume  that  we  are  given  a  finite  set  of  positive  stochastic  tree  samples,  say  V,  and 
another  finite  set  of  negative  stochastic  tree  samples,  say  A f.  We  are  interested  in 
learning  an  LPTS  L  such  that  P  A  L  for  every  P  £  V  and  N  ^  L  for  every  N  £  A f . 
Such  an  L  is  said  to  be  consistent  with  the  given  tree  samples.  Note  that  the  LPTS 
obtained  by  merging  the  start  states  of  all  trees  in  V,  denoted  Lp,  can  be  easily 
shown  to  satisfy  P  A  Lp  for  every  P  £  V .  If  V  =  0,  we  let  Lp  to  be  the  single-state 
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LPTS  with  no  transitions.  Now,  if  L  is  an  arbitrary  consistent  LPTS,  one  can  also 
show  that  Lp  A  L  and  hence,  by  Lemma  13,  Lp  will  also  be  consistent.  Thus,  one 
can  check,  in  polynomial  time,  whether  there  exists  a  consistent  LPTS  by  checking 
N  -<  Lp  for  every  N  G  A f.  However,  the  size  of  Lp  is  equal  to  the  combined  size 
of  all  trees  in  V.  So,  the  question  we  want  to  address  is  whether  we  can  find  small 
consistent  LPTSes. 

As  mentioned  earlier,  given  our  ultimate  objective  of  learning  small  assumptions 
for  compositional  reasoning,  we  are  interested  in  learning  a  consistent  LPTS  of  a 
small  size,  preferably  the  smallest.  To  that  effect,  we  describe  algorithms  that  ob¬ 
tain  a  folding  of  the  tree-shaped  Lp  into  a  consistent  LPTS.  The  algorithms  we 
propose  draw  inspiration  from  state-space  partitioning  techniques  for  obtaining  con¬ 
sistent  automata  from  counterexample  traces  [27,  58,  67,  ].  Let  Sp  =  [J PeVSp 

and  Sjy  =  U.veTV  where  <Sl  denotes  the  set  of  states  of  an  LPTS  L.  First,  we 
consider  an  algorithm  based  on  traditional  state-space  partitioning  of  Sp.  While  this 
approach  does  reduce  the  number  of  states  in  the  inferred  consistent  LPTS,  it  does 
not  guarantee  minimality  in  terms  of  the  number  of  states.  Nevertheless,  as  we  will 
see  in  Section  5.3,  we  find  it  useful  for  the  learning  loop  in  LearnLPTS  (Fig.  5.1) 
to  converge.  We  will  then  introduce  a  new  stochastic  state-space  partitioning  which 
enables  us  to  obtain  a  minimal  consistent  LPTS. 

5.2.1  Using  State-Space  Partitioning 

We  first  describe  an  algorithm  based  on  traditional  state-space  partitioning  of  Sp.  A 
partition  of  a  set  A"  is  a  set  of  non-empty  subsets  of  X  such  that  every  element  of  X 
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is  in  exactly  one  of  the  subsets.  A  partition  II  of  X  induces  an  equivalence  relation 
which  relates  two  elements  iff  they  are  in  the  same  subset,  i.e.,  the  equivalence 
classes  under  the  equivalence  relation  are  nothing  but  the  subsets  in  the  partition. 
For  a  partition  II  of  Sp  and  a  state  s  E  Sp,  we  let  [s]n  denote  the  equivalence 
class  of  s.  Throughout  this  section,  we  assume  that  the  start  states  of  all  positive 
samples  (V)  are  in  the  same  equivalence  class,  i.e.,  [s°P]n  =  [sg]n  for  every  P,Q  G  P. 
Given  a  partition  II,  one  can  obtain  a  quotient  LPTS  where  the  states  correspond 
to  the  equivalence  classes  and  the  distributions  of  the  transitions  of  V  are  lifted  to 
distributions  over  equivalence  classes: 

Definition  13  (Quotient  LPTS).  Given  a  partition  II  of  S-p,  the  quotient  of  V, 
denoted  V/U,  is  the  LPTS  (n,e°,a,r)  where  e°  =  [sp]n  for  every  P  G  V,  a  = 
|J peVctp  and  (e,a,/i)  G  r  iff  there  exists  ( s,a,nP )  G  rP  for  some  P  G  V  with 
[s]n  =  e  and  pp  is  lifted  to  II  to  obtain  p,  i.e.,  p(e')  =  ^s'ee' /h>(s0  for  all  e'  G  II. 
We  use  liftn(pp )  to  denote  the  lifting  of  pp  to  the  partition  II. 

It  is  straightforward  to  show  that  a  quotient  is  a  well-defined  LPTS,  i.e.,  the 
liftings  of  the  distributions  of  V  are  well-defined  distributions  over  equivalence  classes. 
The  following  lemma  shows  that  a  quotient  simulates  every  positive  sample. 
Lemma  20.  Let  II  be  a  partition  of  Sp.  Then,  P  P  V/U  for  every  P  GP. 

Proof.  Let  P  G  V  and  let  P  =  (Sp,  s°P,ap,Tp) .  To  show  that  P  P  V/U ,  consider 
the  binary  relation  R  =  {(s,  [s]n)  |  s  G  Sp}.  Note  that  the  start  state  of  V/U  is 
[sp]n  and  hence,  the  start  states  of  P  and  V/U  are  related  by  R.  It  suffices  to  show 
that  I?  is  a  strong  simulation. 

Let  s  G  Sp  and  s  A  pp  be  arbitrary.  By  definition,  there  exists  a  transition 
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b 


o 

Nb 


Figure  5.2:  Example  stochastic  trees,  divided  into  a  positive  sample  (P)  and  3  negative  samples 
(Na,  Nt,  for  active  learning. 


[s]n  — >  p  in  V/IL  where  p(e)  =  pp(e)  for  all  e  G  II.  It  suffices  to  show  that  pp  p. 
Let  S  C  Supp(fjLp).  We  have, 


Pp('S')  =  J^Pp(^ne) 

egn 

<  ^  /ip(e) 

egn,Sne^0 

=  Me) 

eg  R(S) 

=  Me) 

eg  R(S) 

=  v(R(s)). 

As  S'  is  arbitrary,  it  follows  from  Lemma  11  that  pp  jZ r  p.  We  conclude  that  R 
is  a  strong  simulation.  □ 


{II  is  a  partition} 


(definition  of  i?} 
(definition  of  /i} 
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A  G  (0,  1) 


1  -  A 


H  i 


Figure  5.3:  Quotients  for  least  size  partition  (Hi)  and  stochastic  partition  (H\)  of  P  in  Fig.  5.2. 


If  the  quotient  LPTS  is  also  consistent,  i.e.,  does  not  simulate  any  negative  sam¬ 
ple,  we  call  the  partition  a  consistent  partition.  For  example,  Fig.  5.2  shows  a  positive 
sample  P  and  3  negative  samples  Na,  Nb,  and  for  some  /3, 7  G  (0, 1].  For  these 
samples,  H\  in  Fig.  5.3  is  a  consistent  quotient  LPTS  obtained  from  the  partition 
{{si},{s2},{s3,s4}}  S-p-  The  following  lemma  shows  that  one  can  bound  the 
number  of  states  of  a  consistent  LPTS  obtained  using  the  partitioning  approach. 
Lemma  21.  If  L  is  an  LPTS  of  k  states  that  simulates  all  samples  in  V ,  then  there 
exists  a  partition  II  of  Sp  of  size  at  most  2k  such  that  V/Tl  A  L. 

Proof.  Let  P  G  V.  Let  P  =  (Sp,  s°Pl  ap,Tp)  and  L  =  (Sl,  s°l,o:l,Tl).  We  know 
that  P  <  L.  That  is,  there  exists  a  strong  simulation  Rp  C  Sp  x  Sl  with  s^Rpsf. 
As  P  is  a  tree,  s°P  is  not  in  the  support  of  any  distribution  and  hence,  Rp(s°P )  does 
not  affect  whether  Rp  is  a  strong  simulation  or  not.  So,  without  loss  of  generality, 
assume  that  Rp(s°P )  is  the  singleton  {s°L}.  Let  R  =  (J PeVRp.  Now,  R  induces  an 
equivalence  relation  E  over  Sp  such  that  S\Es2  iff  R(s\)  =  i?(s2).  Let  II  be  the 
partition  corresponding  to  E.  Note  that  [s°P]n  =  [sq]u  for  P,Q  G  V,  satisfying  our 
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assumption  on  a  partition  that  the  start  states  of  all  samples  in  V  are  in  the  same 
equivalence  class.  The  size  of  II  is  clearly  bounded  by  2k. 

In  order  to  show  V/Tl  P  L,  consider  the  binary  relation  R'  =  {([sp  ]n,  si)  | 
spRsi,sp  G  S-pjSi  G  Sl}  between  the  states  of  V/Tl  and  L.  Clearly,  R'  relates 
the  start  state  of  V/Tl,  say  n,  with  s°,  as  s°PRs°L  for  every  P  E  V.  It  suffices  to 
show  that  R'  is  a  strong  simulation. 

Let  eR'si  and  e  A  fi  be  arbitrary.  By  Definition  13,  there  exists  sp  G  S-p  and 
fip  G  Dist(S-p)  with  [sp]n  =  e,  sp  -A  nP  and  /r(e')  =  /xp(e')  for  all  e'  G  E.  Furthermore, 
by  the  definitions  of  R'  and  II,  spRsi.  As  R  is  a  strong  simulation,  there  exists 
Hi  G  Dist(Si)  such  that  si  A  fii  and  HP  ^=R  /h-  Let  E'  C  Supp(n)-  Now, 


v(E')  =  ^ 


e'GE’ 


=  aA') 


e'eE1 


e'&E' 


~Hp{{s  G  Sp  |  [s]n  G  E1}) 
<Hi(R({s  G  Sv  |  [s]u  G  £'})) 
=/h(  U  -R({s  €  A5  I  [s]n  =  e'})) 


{/A  !=r?  Hi} 


e'GE1 


{Def.  of  R'} 


e'eE1 


Hi{V(E')). 
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As  E'  is  arbitrary,  it  follows  from  Lemma  11  that  fi  ER>  /ip  We  conclude  that  R! 
is  a  strong  simulation.  □ 

Note  that,  if  L  and  every  P  G  V  is  non-probabilistic  (i.e. ,  an  LTS),  then  one 
can  always  choose  the  strong  simulation  Rp  in  the  above  proof  to  be  a  function  and 
the  bound  in  the  above  lemma  goes  down  to  k.  The  following  is  immediate,  using 
Lemmas  13  and  20. 

Corollary  3.  For  every  consistent  LPTS  of  k  states,  there  is  a  consistent  partition 
of  size  at  most  2k. 

In  other  words,  the  state-space  partitioning  approach  for  learning  a  consistent 
LPTS  can  be  at  most  exponentially  worse,  in  terms  of  the  number  of  states.  While 
this  is  only  an  upper  bound,  we  can  also  show  that  this  approach  cannot  guarantee 
minimality  in  general.  To  see  this,  H\  in  Fig.  5.3,  for  any  A  G  (0,1),  is  also  a 
consistent  LPTS  for  the  samples  in  Fig.  5.2  with  one  less  state  when  compared  to 
H\ ,  the  smallest  LPTS  one  can  obtain  using  state-space  partitions.  This  is  surprising 
at  first,  as  it  is  well  known  for  non-probabilistic  systems  and  trace  counterexamples 
that  there  always  exists  a  consistent  partition  of  the  least  number  of  states  [67,  96]. 

Algorithm 

A  naive  algorithm  for  finding  a  least-sized,  consistent  partition  is  to  enumerate  all  the 
partitions  of  Sp  for  increasing  values  of  the  size,  and  for  each  of  them,  check  if  the 
corresponding  quotient  simulates  any  tree  in  A f.  Alternatively,  in  the  case  where  all 
the  probabilities  involved  are  rational,  we  can  utilize  the  efficient  solvers  that  exist 
today  for  satisfiability  modulo  theories  (SMT),  and  in  particular,  for  the  theory  of 
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1 

2 

3 


EncodeConsisPartition('P,  Af,  k ) 

introduce  Boolean  variables  IISij  to  denote  [s]n  =  ej1  for  (s, i)  £  Sp  x  {1, . . .  ,k} 

for  s  £  S-p  do 

|_  ADDCONS(xor(nS;i,...,nSife)) 


4 

5 

6 

7 


for  P  £  V  do 

|  AddCONS(IIs03i1) 

for  N  £  Af  do 

//  encode  N  ^  V /H 
EncodeNotSim(A,  V,  II,  k) 


Figure  5.4:  SMT  encoding  for  a  consistent  partition  of  size  k  for  samples  V  and  A f. 


linear  rational  arithmetic,  as  shown  below.  We  expect  this  to  be  more  efficient  than 
an  exhaustive  search,  in  practice.  Moreover,  this  prepares  the  ground  for  an  optimal 
algorithm  we  discuss  in  the  next  subsection. 

As  mentioned  at  the  beginning  of  the  section,  we  can  easily  check  if  there  exists 
a  consistent  LPTS  by  merging  the  start  states  of  all  positive  samples  to  obtain  LP 
and  checking  if  Lp  is  consistent.  If  there  exists  a  consistent  LPTS,  we  can  search  for 
the  smallest  consistent  partition  by  iteratively  checking  if  there  exists  a  consistent 
partition  of  size  k,  for  increasing  values  of  k.  For  a  given  k,  we  encode  the  existence 
of  a  consistent  partition  of  size  k  as  an  SMT  problem  using  EncodeConsisPar- 
TITIOn(”P,  A/",  k)  in  Fig  5.4.  Here,  we  introduce  Boolean  variables  to  denote 
[s]n  =  ej1  for  some  partition  n  =  {e^1, . . . ,  e ]?}  of  size  k.  The  constraints  added  on 
lines  3  and  5  essentially  encode  that  the  partition  n  is  well-defined,  i.e.,  each  state 
s  £  Sp  belongs  to  exactly  one  element  of  the  partition  and  the  start  states  of  all 
samples  in  V  belong  to  the  same  equivalence  class  e^,  respectively. 

For  n  to  be  consistent,  we  need  to  encode  that  no  sample  in  A f  is  simulated  by 
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1 

2 

3 

4 


EncodeNotSim(1V,  V ,  n,  k) 

introduce  Boolean  variables  RSn,i  to  denote  (sn,ep)  €  R  C  Sjv  x  II 
introduce  Boolean  variables  relPn^p  to  denote  /j,n  C#  Hflui^p) 
for  erery  (s,i)  E  SV  x  {!,•■■,*}  do 
Add  Cons  (-RSri)j  <«=*> 

^{(a,/i„)|s„— >Hn)  ^ {(sp,fip)\sp&S-p,sp— >fj,p}  (^sp>®  ^  re^‘n,Mp)) 


5 

6 

7 


for  every  rel^.,^  do 

j_  EncodeDistRel^,  liftn(np),  R,  relPn^p) 
AddCons  (^Rson1) 


Figure  5.5:  SMT  encoding  for  N  ^  V/H. 


the  quotient  V /H  (line  7).  A  naive  encoding  introduces  a  universal  quantification 
over  all  possible  strong  simulations  to  say  that  no  strong  simulation  relates  the  start 
states  of  a  sample  in  A f  and  V /li.  We  can  avoid  this  by  using  the  characterization  of 
A  in  Lemma  14  for  trees,  as  shown  in  Fig.  5.5.  Here,  we  introduce  Boolean  variables 
RSn,i  to  denote  snReY  for  the  coarsest  strong  simulation  R  between  the  tree  N  and 
V/H  and  reltln)lJp  to  denote  /xn  Qr  tiftnil-b)  f°r  distributions  /x„  G  -Dzst(Av)  and  nP  G 
Dist(S-p).  The  constraints  added  on  line  4  essentially  encode  the  characterization 
of  A  in  Lemma  14.  In  words,  snReji  holds  iff  for  every  transition  sn  A  /xn,  there 
exists  some  transition  sp  A  /xp  in  V  on  the  same  action  a  such  that  sp  belongs  to 
the  equivalence  class  ej1  and  the  lifting  of  /up  is  related  to  /in.  The  constraint  on 
line  7  encodes  that  the  coarsest  strong  simulation  does  not  relate  the  start  states  of 
N  and  V /H.  Finally,  we  encode  the  constraints  on  the  variables  relPnyPp  as  described 
in  Chapter  4  (See  Fig.  4.10).  This  needs  us  to  encode  the  lifting  liftn(nP)  of  a 
distribution  [ip  to  n,  which  we  do  as  follows. 


Given  jip  G  Dist(S-p)  and  an  equivalence  class  ej1,  liftn (//p)(ef)  can  be  encoded 
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as  ^sQSuppifip)  ^aw,s  wliere  ^nP,i,s  is  a  rational  variable  denoting  the  contribution  of 
s  towards  the  probability  of  the  equivalence  class  ej1  in  the  lifted  distribution,  for 
which  we  add  the  constraints: 

(nSii  v  /^p(®))  ^  (  1 y  & nP,i,s  o)- 

The  following  is  immediate. 

Lemma  22.  There  exists  a  consistent  partition  of  size  k  for  samples  V  and  J\f  iff 
the  constraints  resulting  from  EncodeConsisPartition(7:,,.A/’,  k)  are  satisfiable. 

5.2.2  Using  Stochastic  State-Space  Partitioning 

Consider  again  the  positive  and  negative  tree  samples  in  Fig.  5.2.  As  mentioned  in 
the  previous  subsection,  H\  in  Fig.  5.3  is  the  smallest  (w.r.t.  the  number  of  states) 
consistent  LPTS  that  can  be  obtained  using  the  state-space  partitioning  approach, 
but  H\  in  the  figure  is  also  consistent  with  one  less  state.  We  can  also  show  that 
there  is  no  consistent  LPTS  with  fewer  states  than  H\  -  in  order  to  simulate  the 
positive  sample,  an  LPTS  with  a  single  state  should  have  self  loops  on  all  3  actions 
which  would  then  simulate  all  the  negative  samples  as  well.  To  be  able  to  fold  the 
positive  sample  P  into  H\,  we  need  a  way  to  group  the  states  of  P  such  that  there  is 
a  one-to-one  correspondence  between  the  states  and  the  transitions  of  the  folding  and 
H\.  As  we  have  seen  with  the  above  example,  the  state-space  partitioning  approach 
does  not  guarantee  that  and  Lemma  21  shows  an  exponential  upper  bound  on  the 
number  of  states  of  the  resulting  folding.  In  this  subsection,  we  will  describe  an 


154 


S1 


Figure  5.6:  LPTS  obtained  by  splitting  s2  of  P  in  Fig.  5.2  into  s2i  and  s22- 

alternative  approach  for  folding  the  states  to  obtain  consistent  LPTSes  of  the  least 
number  of  states. 

We  start  with  a  high  level  description  of  the  approach  using  the  above  mentioned 
example.  Let  R  be  a  strong  simulation  between  the  positive  sample  P  (Fig.  5.2)  and 
H\  (Fig.  5.3).  Let  the  states  of  the  LPTSes  be  labeled  as  shown  in  the  figures.  It  is 
not  hard  to  show  that  R  must  relate  s2  to  both  t\  and  t2  in  order  for  the  transition 
on  a  in  P  to  be  simulated  by  H\.  Now,  consider  the  LPTS  P'  in  Fig.  5.6  obtained 
from  P  by  splitting  s2  into  two  states  S21  and  s22  such  that  the  transition  from  Sj 
on  action  a  leads  to  S21  with  probability  1  —  A  and  s22  with  probability  A.  We  can 
easily  show  that  P  is  simulation  equivalent  to  P'  (i.e.,  P  P  P'  and  P'  -<  P)  and 
so,  we  can  consider  P'  as  the  positive  sample,  instead  of  P .  But,  more  importantly, 
we  can  now  obtain  a  partition  II'  =  {{si,  S21},  {S22,  S3,  S4}}  of  the  state-space  of  P' 
whose  quotient  is  exactly  H\. 

Alternatively,  the  above  state  splitting  can  be  understood  in  the  following  way,  in 
terms  of  the  state-space  of  P.  For  each  state  of  P,  we  assign  a  probability  distribution 
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over  a  finite  set,  whose  elements  we  call  groups,  as  opposed  to  assigning  a  unique 
equivalence  class  in  the  case  of  a  partition.  For  example,  corresponding  to  the  state 
splitting  mentioned  above,  we  have  two  groups,  say  gi  and  g2l  where  S2  is  assigned  the 
distribution  which  has  probability  1  —  A  for  gi  and  probability  A  for  g2 ,  s  1  is  assigned 
the  distribution  Sgi,  and  S3  and  S4  are  assigned  the  distribution  Sg2.  In  general,  the 
distribution  associated  with  a  state  (in  other  words,  the  splitting  of  a  state)  in  the 
positive  sample  depends  on  the  current  group  (in  other  words,  the  current  split)  of 
its  parent.  We  formalize  these  ideas  as  a  stochastic  partition  of  Sp,  defined  below. 
For  s  G  S-p,  we  write  par(s)  to  denote  the  unique  parent  of  s. 

Definition  14  (Stochastic  Partition).  A  stochastic  partition  II  of  Sp  is  a  tuple 
( G,g°,D )  where  G  is  a  finite  set  whose  elements  are  called  groups,  g°  G  G,  and 
D  :  S  — >■  (G  — ^  Dist(G)),  such  that  the  following  hold.  Let  s  G  Sp  and  g  G  G  be 
arbitrary. 

1.  if  s  is  a  start  state,  D{s){g)  =  6go,  and 

2.  if  s  is  a  not  a  start  state,  D(s)(g)  is  defined  iff  g  G  Supp (D(par(s))(h))  for 
some  h  G  G. 

We  write  [s]n  to  denote  the  distribution  map  D(s).  When  II  is  clear  from  the  context, 
we  drop  the  subscript.  For  convenience,  we  write  s  G  g  to  denote  g  G  Supp (D(s)(h)) 
for  some  h  G  G  and  we  sometimes  confuse  g  with  the  set  of  all  states  s  such  that 
s  e  g. 

Intuitively,  a  distribution  assigned  to  a  state  s  by  a  stochastic  partition  specifies 
how  s  and  its  incoming  transitions  are  split  which  depends  on  how  its  parent  was 
split.  To  see  how  to  fold  the  positive  samples,  given  a  stochastic  partition,  we  need 
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to  define  its  quotient ,  analogous  to  the  quotient  of  a  partition.  Note  that,  for  a 
state  s  and  a  group  g,  [s]n(<?)  is  n°t  always  defined.  For  convenience,  we  extend  [s]n 
into  a  total  function  by  letting  [s]n(fiO(A  —  0  for  every  group  h,  whenever  [s]n(<?)  is 
undefined. 

Definition  15  (Quotient  LPTS).  Given  a  stochastic  partition  II  =  (G,g°,D)  of  S-p, 
the  quotient  of  II,  denoted  V/H,  is  the  LPTS  (G,  g° ,a,r)  where  a  =  |J PeVap  and 
(g,  a,  /i)  G  r  iff  there  exists  (s,  a,  fip)  G  Tp  for  some  P  G  V  with  s  G  g  and  fip  is  lifted 
to  fl  to  obtain  //  as  follows:  for  every  g'  G  G, 


MX)  =  (M]n(M(X)  ‘  Ms'))  • 

s' eg' 

We  write  liftUg(iip )  to  denote  the  lifting  of  fip  to  fl. 

In  other  words,  given  s  A  /ip,  the  lifting  of  gp  assigns  a  probability  to  a  group  g' 
that  is  equal  to  the  sum  of  the  probabilities  of  all  states  under  pp  weighted  by  the 
probabilities  of  them  being  assigned  to  g'  under  II.  Moreover,  this  is  in  the  context 
of  the  parent  s  being  assigned  to  a  group  g.  For  instance,  consider  the  transition 
Si  A  SS2  of  the  positive  sample  P  in  the  above  example.  Note  that  g\  =  {si,  S2}  and 
moreover,  rq  is  the  only  group  containing  Si.  The  probability  of  g\  under  the  lifting 
of  the  distribution  of  this  transition  is  obtained  as 

(M^iX^i)  •  Ms))  =  A] (91) (91)  ■  s8a(si)  +  [s2] AO (gi)  ■  sS2(s2 ) 

s&g  1 

=  0  +  (1  —  A)  •  1 
=  1- A 
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Similarly,  one  can  obtain  the  probability  of  g2  under  the  lifting  as  A.  After  lifting 
all  the  transitions  of  P  in  this  way,  one  can  see  that  the  quotient  is  essentially  H\ 
where  t\  and  t2  in  Fig.  5.3  correspond  to  gi  and  g2,  respectively. 

Before  moving  on,  we  show  that  the  quotient  is  a  well-defined  LPTS,  i.e. ,  the 
lifting  of  a  distribution  to  a  stochastic  partition  is  a  well-defined  distribution  over  its 
groups. 

Lemma  23.  Let  U  be  a  stochastic  partition  of  S-p.  Then,  V /H  is  a  well-defined, 


LPTS. 


Proof.  Let  G  be  the  set  of  groups  of  II  and  let  g  A  p  be  a  transition  of  V /H.  It 
suffices  to  show  that  p  G  Dist(G).  From  Definition  15,  there  exists  s  A  pp  for  s  G  S-p 
such  that  s  G  g  and  p  =  liftU  g(pp).  Now, 


g'&G 


g'GGs'eg' 


{Definition  14} 


{[site)  e  DisfiG)} 


s'eSr 


1 


{pp  G  DisfiSp)} 


□ 


We  have  the  following  lemma  analogous  to  state  partitions. 
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Lemma  24.  Let  U  be  a  stochastic  partition  of  S-p.  Then,  P  S  V /H  for  every  P  E  V . 

Proof.  Let  II  =  (G,g°,D)  and  let  P  =  (Sp,sP,ap,Tp)  be  a  sample  in  V.  Consider 
the  relation  R  =  {(s,g)  \  g  E  G,s  E  Sp  Dp}.  By  Definition  14,  sPRg°.  To  show 
P  Si  V / n,  it  suffices  to  show  that  R  is  a  strong  simulation. 

Let  sRg  and  s  A  gp.  As  s  E  g,  by  Definition  15,  g  A  g  where  g  =  liftUg(gp). 
We  will  now  show  that  np  g.  Let  S  C  Supp(gp).  We  have 


Tp(S)  =  ^/ip(s') 
s'eS 

=  E  E  (M(9)(9')-^M)  (M(S)  £  «(«)} 

s'eS{g'\s'eg'} 

=  E  E  (MWW-fcM) 

g'&Gs'&Sng' 

=  E  E  ([s']  (#)(#')  •  l-h(s'))  {Definition  of  R} 

g'eR(S)  s'GSCg ' 

<  E  E(M(9)(9')'ft>(s')) 

g'&R(S)  s’e g' 

=  hid)  {Definition  15} 

g'GR(S) 

=  i*(R(S)) 


So,  by  Lemma  11,  gp  g. 
strong  simulation. 


As  s  A-  gp  is  arbitrary,  we  conclude  that  R  is  a 

□ 
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We  will  now  show  that  stochastic  partitioning  results  in  consistent  LPTSes  of  the 
least  number  of  states. 

Lemma  25.  If  L  is  an  LPTS  of  k  states  that  simulates  all  samples  in  V ,  then  there 
exists  a  stochastic  partition  II  of  Sp  of  k  groups  with  V /IIP  L. 

Proof.  Let  P  G  V.  Let  P  =  ( Sp ,  sP,ap,  rp)  and  L  =  ( Sl ,  s°L,  oil,  tl)-  We  know  that 
PPL.  That  is,  there  exists  a  strong  simulation  Rp  C  Sp  x  Sl  with  s°PRps°L.  Now, 
let  R  =  U P<zpRp-  Let  sp  G  Sp  and  si  €  Sl-  We  assume  a  choice  function  Witness 
that  given  a  pair  (sp  -A  p,p,si)  with  spRsi ,  outputs  {ni,w)  such  that  Si  A  pLi  and  w 
is  a  weight  function  witnessing  fip  C#  /i;  according  to  Definition  8.  Such  a  choice 
function  always  exists  given  that  R  is  a  strong  simulation. 

Let  sp  G  SP  for  some  P  G  V.  We  define  the  depth  of  sp  as  its  distance  from 
the  start  state  s°P.  We  define  a  stochastic  partition  II  =  (G,g°,D)  where  there  is 
a  one-to-one  correspondence  7  between  Sl  and  G  such  that  7 (s°L)  =  g°  and  D  is 
defined  as  follows  by  induction  on  the  depth  of  a  state  sp  G  Sp.  Using  the  same 
induction,  we  also  show  the  following  properties.  Let  si  G  Sl  and  g  G  G. 

1.  sp  G  'y(si)  iff  spRsi  holds, 

2.  there  exists  an  h  G  G  such  that  sp  G  /i, 

3.  if  sp  is  a  start  state,  D(sp)(g)  =  5S 0 ,  and 

4.  if  sp  is  not  a  start  state,  D(sp)(g)  is  defined  iff  par(sp )  G  g. 

In  the  base  case,  sp  is  a  start  state  and  we  define  D(sp)(g)  to  be  the  Dirac 
distribution  <5so  for  every  g  G  G.  It  is  easy  to  see  that  the  4  properties  mentioned 
above  are  satisfied  for  sp. 
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In  the  inductive  case,  there  is  a  unique  transition  par(sp )  A  pp  such  that  sp  G 
Supp(gp).  Let  g  G  G  be  arbitrary  such  that  par(sp )  G  g  and  let  g  =  7 (£/).  Such 
a  group  g  is  guaranteed  to  exist  by  inductive  hypothesis.  We  now  define  D(sp)(g) 
for  every  such  group  g.  Note  also  that,  by  inductive  hypothesis,  par(sp)Rti.  Let 
Witness  (par  (sp)  A  pp,ti)  =  (pi,w).  Then,  we  know  that  ti  A  pi  and  the  weight 
function  w  witnesses  pp  pd .  We  then  define  D(sp)(g )  to  be  a  distribution  p  G 
Dist(G )  such  that  p(nf(si))  =  w(sp,  si)/ pp(sp)  for  every  s/  G  SA  It  follows  from 
Definition  8  that  n(l(si))  =  M-A,  A  /Msp))  =  1  and  hence,  p  is 

well-defined.  It  is  also  easy  to  see  that  the  4  properties  mentioned  above  are  satisfied 
for  sp. 

The  4  properties  mentioned  above  immediately  imply  that  II  is  a  well-defined 
stochastic  partition,  i.e. ,  II  satisfies  all  the  requirements  of  Definition  14. 

We  will  now  show  that  V/Tl  L.  Consider  the  binary  relation  R!  =  {(g,s/)  | 
g  =  7 (si)}.  Clearly,  ( g°,s°L )  G  i?'  by  construction  of  II.  It  suffices  to  show  that  R'  is 
a  strong  simulation  between  V /li  and  L. 

Let  ( g,si )  G  R!  and  g  A  p.  Then,  there  exists  sp  G  g  such  that  sp  A  and 
g  =  liftng(pp)-  By  construction  of  II,  we  know  that  spRsi.  Let  Witness(sp  A  gp,  s/) 
output  (pRw)  such  that  si  A  pi  and  w  is  a  weight  function  witnessing  pp  \ZR  ph  We 
will  show  that  p  Ar'  Pi-  Let  g'  G  Supp(p)  and  g'  =  7 (s^).  We  have, 

Ms /)  =  (MteXs')  •  Ms')) 

s'&g' 

=  X  (D(s')(g)(l(s'i))  -Ms'))  {notation} 

s'e7(s|) 
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{construction  of  11} 


=  Y1  u'(s'’s0 

s'€7(s{) 

=  u)(s',  s[)  {property  1  above} 

s'Rs't 

=  ni( s \)  {Definition  8} 

=  MW)) 

As  g'  is  arbitrary,  it  follows  from  Lemma  11  that  fi  /q .  We  conclude  that  R'  is 
a  strong  simulation.  □ 

The  basic  intuition  behind  the  above  lemma  is  that  one  can  associate  a  group 
with  each  state  in  Sl  and  the  weight /flow  function  that  witnesses  gp  FR  gi  for 
jip  G  Dist(Sp)  and  Hi  G  Dist(Si)  identifies  a  splitting  of  the  probabilities  under 
gp  to  the  states  in  Supp(gi)  C  Sl-  This  splitting  can  then  be  used  to  define  the 
distributions  of  a  stochastic  partition  whose  quotient  is  also  consistent. 

Our  main  result  is  immediate,  using  Lemmas  13  and  24.  As  in  the  case  of  parti¬ 
tions,  we  say  that  a  stochastic  partition  IT  is  consistent  iff  V /H  is  consistent. 
Corollary  4.  For  every  consistent  LPTS  of  k  states,  there  is  a  consistent  stochastic 
partition  of  k  groups. 

Algorithm 

As  in  the  previous  subsection,  we  can  search  for  a  consistent  stochastic  partition 
of  the  least  size  by  iteratively  checking  if  there  exists  a  consistent  stochastic  parti¬ 
tion  of  size  k,  for  increasing  values  of  k.  Assuming  that  all  probabilities  involved 
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1 


2 


3 


EncodeConsisStochPartition(P,  Af,  k) 

introduce  non-negative  rational  variables  Il^jj  to  denote  [s] n  )  ( f/j1 )  for 

£  Sv  x  {1, . . k}  X  {1,  ...,k} 
for  s  £  S-p  and  1  <  i  <  k  do 

//  [s]n(ffi)  is  either  well-defined  or  undefined 

_  AddCons((Ei <j<kUs,iJ  =  l)  V  1  <j<k  —  o) ) 


4 

5 


for  PeP  and  1  <  i  <  k  do 

//  Condition  1  of  Definition  14 

Add  Cons  =  1) 


6 

7 

8 

9 


for  every  non-start  state  s  £  S-p  and  1  <  i  <  k  do 
//  Condition  2  of  Definition  14 

_  ADDCONS(Ei<j<fen s,ij  =  1  <<=►  El<j<kUpar(s),j,i  >  0) 

for  N  £  Af  do 

//  encode  N  ^  V /J1 
EncodeNotSim(A,  V.  II,  k) 


Figure  5.7:  SMT  encoding  for  a  consistent  stochastic  partition  of  size  k  for  samples  V  and  Af. 


are  rational,  we  can  again  encode  each  iteration  as  an  SMT  problem  using  ENCODE- 
ConsisStochPartition(‘P,  A\f,  k )  in  Fig.  5.7.  Here,  we  introduce  rational  variables 
nSjjj  to  denote  gf)  for  some  stochastic  partition  n  =  ({g]1, . . . ,  gj}},  g]1,  D). 

The  constraints  on  lines  3,  5,  and  7  essentially  encode  that  the  stochastic  partition 
n  is  well-defined. 

Encoding  consistency,  i.e.,  that  no  sample  in  J\f  is  simulated  by  the  quotient 
V /n,  is  similar  to  EncodeNotSim  in  Fig.  5.5  except  that  the  equivalence  classes 
are  replaced  by  the  groups  of  n  and  ns  i  on  line  4  is  replaced  by  Yh\<3<k  H.sj,*  >  0. 
Moreover,  given  /ip  £  Dist(Sp)  and  groups  gf  and  gj1,  liftn ,gn(/ip)(gjI)  is  encoded  as 

Y2s£Supp(iip)  •  /ip(S))- 

The  following  is  immediate. 
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Theorem  13.  There  exists  a  consistent  stochastic  partition  of  size  k  for  samples  V 
and  M  iff  the  constraints  resulting  from  ENCODECONSISSTOCHPARTITION^,  J\f,  k ) 
are  satisfiable. 

5.3  Convergence  in  Active  Learning 

Now  that  we  have  discussed  algorithms  for  inferring  an  LPTS  consistent  with  a  given 
set  of  positive  and  negative  samples  (FindConsistent),  we  turn  our  attention  to 
the  convergence  of  the  active  learning  framework  LearnLPTS  (see  Fig.  5.1).  As  we 
have  seen  in  Section  5.1,  each  iteration  of  LearnLPTS  infers  a  consistent  LPTS  for 
the  tree  samples  returned  by  the  teacher  so  far  with  the  ultimate  goal  of  converging 
to  an  LPTS  that  is  (simulation)  equivalent  to  the  unknown  target.  We  start  with 
a  negative  result  which  shows  that  under  no  assumptions  about  the  samples  the 
teacher  can  return,  there  is  no  converging  solution  to  the  learning  problem. 
Theorem  14.  There  is  no  converging  learning  algorithm  in  the  active  learning  frame¬ 
work  LearnLPTS. 

Proof.  Consider  the  LPTS  U\  in  Fig.  5.9,  parametric  in  a  rational  number  A  G  (0, 1). 
As  shown  in  the  figure,  U\  has  one  transition  from  the  start  state  on  a  leading  to  a 
distribution  /i\.  Fig.  5.8  shows  an  adversarial  teacher  that  manipulates  the  value  of 
A  dynamically,  as  necessary,  to  ensure  that  a  counterexample  always  exists  no  matter 
what  LPTS  the  learner  conjectures  on  line  4.  Moreover,  A  is  updated  in  such  a  way 
that  the  new  U\  remains  consistent  with  all  the  previously  generated  samples.  This 
will  ensure  that  the  learner  never  converges.  We  describe  the  teacher  below. 
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AdversarialTeacherQ 

1  A  arbitrary  rational  in  (0, 1) 

2  M  0 

3  while  true  do 

4  H  i-  GetNextConjecture() 

5  if  H  U\  then 

6  obtain  a  tree  counterexample  N  and  add  to  JV 

7  return  N  as  a  negative  sample 


8 

g 

10 


else  if  U\  ^  H  then 

obtain  a  tree  counterexample  P 
return  P  as  a  positive  sample 


11 

12 

13 

14 

15 


else 

A+  =  min  ({p£  |  p%  >  A,  v  G  Dist[a,J\f]}  U  {1}) 
//  see  text  for  description 
A  (A+  +  A)/2 

obtain  a  tree  counterexample  P  to  U\  ^  H 
return  P  as  a  positive  sample 


Figure  5.8:  An  adversarial  teacher  in  the  proof  of  Theorem  14. 


For  every  new  conjecture  made  by  the  learner,  the  teacher  first  checks  if  there 
is  a  counterexample  w.r.t.  U\,  for  the  current  value  of  A,  and  returns  a  positive  or 
negative  sample,  as  appropriate  (lines  4-10).  When  the  conjecture  H  results  in  no 
counterexamples,  it  increases  the  value  of  A  as  follows.  Let  Dist[a,N\  =  {u  \  s°N  A 
v  for  some  N  G  W}  be  the  set  of  all  distributions  of  the  transitions  on  action  a 


Figure  5.9:  A  target  in  the  active  learning  framework  where  an  adversarial  teacher  can  dynamically 
modify  the  probability  A  leading  to  the  divergence  of  a  learner. 
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outgoing  from  a  start  state  in  A f.  Given  v  G  Dist[a, A/"],  let  p%  be  the  cumulative 
probability  under  v  of  all  states  in  Supp(u)  that  have  at  least  one  outgoing  transition 
on  action  b.  The  idea  is  to  consider  the  smallest  of  all  such  p^s  that  are  larger  than 
A  and  to  increase  A  to  a  value  in  between,  say  the  average  (lines  12-13).  If  there  is 
no  ph  that  is  greater  than  A,  we  take  the  average  between  A  and  1.  As  A  is  rational, 
this  is  always  possible  and  the  new  value  of  A  remains  in  (0, 1).  It  remains  to  show 
that  U\,  for  the  new  value  of  A,  remains  consistent  with  all  the  previously  generated 
samples. 

Let  A'  be  the  new  value  of  A.  By  construction,  A'  >  A  and  hence,  U\  Uy.  It 
follows  from  Lemma  13  that  Uy  simulates  every  positive  sample  in  V. 

Let  A  be  a  negative  sample  in  A f.  Assume,  for  the  sake  of  contradiction,  that 
N  -<  Uy.  As  the  only  outgoing  transition  from  the  start  state  of  Uy  is  on  action 
a,  every  outgoing  transition  from  the  start  state  s°N  of  N  must  also  be  labeled  by 
a.  Let  s%  A-  o.  Given  our  assumption  that  N  <  Uy ,  we  have  that  v  0^  py.  We 
can  similarly  reason  that  no  state  in  Supp(y )  has  a  transition  labeled  by  an  action 
other  than  b.  Moreover,  every  other  transition  in  the  tree  N  must  also  be  labeled 
by  b  ioi  N  <  Uy  to  hold.  Consider  p%,  the  cumulative  probability  under  v  of  all  the 
states  in  Supp(v )  that  have  an  outgoing  transition  on  b.  It  follows  from  Lemma  11 
that  pi  <  A'.  Given  that  AMs  a  negative  sample,  N  U\  and  hence,  pvh  >  A.  But 
then,  from  the  above  construction,  A'  <  pvh  holds  which  leads  to  a  contradiction.  We 
conclude  that  N  Uy.  0 

At  a  high  level,  the  above  theorem  holds  because  it  is  not  necessary  for  the 
positive  tree  samples  returned  by  the  teacher  to  have  an  execution  mapping  to  the 
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target  U  (see  Section  4.5).  As  we  have  seen  in  the  proof  of  the  above  theorem,  this 
allows  the  possibility  of  adversarial  behavior  by  deliberately  choosing  the  probability 
values  in  the  samples  such  that  the  learner  is  guaranteed  not  to  converge.  But, 
in  practice,  to  be  able  to  apply  the  learning  framework  in  a  given  setting,  we  also 
need  to  implement  the  teacher  and  we  are  not  aware  of  any  algorithm  to  generate 
counterexamples  other  than  the  one  discussed  in  Section  4.5.  As  mentioned  before, 
this  algorithm  does  have  the  property  that  the  generated  counterexample  to  L\  P  L2 
has  an  execution  mapping  to  L\.  This  suggests  us  to  impose  the  following  friendliness 
condition  on  a  teacher. 

Condition  1  (Friendly  Teacher).  Every  positive  ( negative )  sample  returned  by  the 
teacher  should  have  an  execution  mapping  to  the  target  (conjecture) . 

First  of  all,  note  that  the  proof  of  the  above  theorem  no  longer  works  because  up¬ 
dating  the  value  of  A  in  Fig.  5.9  violates  the  above  condition  on  previously  returned 
positive  samples.  In  fact,  as  we  show  below,  using  the  state-space  partitioning  tech¬ 
nique  for  inferring  a  consistent  LPTS  in  each  iteration  (see  Section  5.2.1)  ensures 
convergence. 

Lemma  26.  Under  Condition  1  on  the  teacher,  the  learning  algorithm  that  computes 
the  quotient  of  a  least-sized  consistent  state-space  partition  leads  to  convergence  in 
the  active  learning  framework  LearnLPTS. 

Proof.  Let  U  be  the  unknown  target  LPTS.  Consider  an  arbitrary  iteration  of  the 
learning  loop  in  LearnLPTS.  First  of  all,  the  execution  mappings  between  the 
positive  samples  and  U  (which  exist  due  to  Condition  1)  induce  an  equivalence 
relation  among  the  states  in  S-p  where  two  states  are  related  iff  they  are  mapped  to 
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the  same  state  of  U .  One  can  easily  show  that  the  quotient  of  the  corresponding 
state-space  partition  is  a  sub-structure  of  U,  i.e.,  U  with  some  (if  any)  transitions 
removed,  and  hence,  is  trivially  simulated  by  U .  As  U  itself  is  a  consistent  LPTS, 
the  quotient  is  also  consistent  (follows  from  Lemma  13).  Therefore,  the  quotient  of 
a  consistent  partition  of  the  least  size  has  at  most  \Su\  number  of  states. 

Now,  one  can  show  that  there  are  only  finitely  many  possible  conjectures  for  a 
given  number  of  states,  across  all  iterations  of  LearnLPTS.  This  is  because,  from 
Condition  1,  every  distribution  in  V  is  a,  replica  of  some  distribution  of  U,  which  are 
finitely  many,  and  lifting  a  distribution  to  a  partition  only  adds  probabilities,  which 
can  only  be  done  in  finitely  many  ways.  As  every  conjecture  is  inconsistent  with 
the  next  sample,  no  two  conjectures  are  simulation  equivalent,  and  hence,  no  two 
conjectures  are  identical. 

Together,  we  conclude  that  the  learner  converges.  □ 

The  following  is  immediate. 

Theorem  15.  There  exists  a  converging  learning  algorithm  in  the  active  learning 
framework  under  Condition  1  on  the  teacher. 

With  our  ultimate  objective  of  deploying  the  learning  algorithm  for  assume- 
guarantee  compositional  reasoning,  it  is  desirable  to  learn  an  LPTS  with  as  few 
states  as  possible.  For  this  purpose,  we  will  now  impose  the  following  condition  on 
the  learner  to  output  only  consistent  LPTSes  of  the  least  number  of  states. 
Condition  2  (Optimal  Learner).  Every  conjecture  H  made  by  the  learner  is  a  con¬ 
sistent  LPTS  of  the  least  number  of  states. 

However,  there  exists  no  converging  learning  algorithm  under  both  Condition  1 
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and  Condition  2,  as  shown  below. 

Theorem  16.  There  is  no  converging  learning  algorithm  in  the  active  learning  frame¬ 
work  LearnLPTS  under  both  Condition  1  on  the  teacher  and  Condition  2  on  the 
learner. 

Proof.  Let  the  LPTS  H\  in  Fig.  5.3  be  the  unknown  target.  We  will  show  that  there 
is  an  adversarial  strategy  for  the  teacher  to  ensure  that  there  is  a  counterexample 
for  every  conjecture  made  by  the  learner.  In  fact,  we  show  that  H\  in  Fig.  5.3,  for  a 
suitable  value  of  A,  is  a  valid  conjecture  for  every  iteration  of  the  learning  loop  for 
the  adversarial  strategy. 

By  Condition  2,  the  learner  only  conjectures  least-state  consistent  LPTSes  and 
so,  the  initial  strategy  of  the  teacher  is  to  return  samples  until  a  1-state  LPTS  H* 
with  self-loops  on  actions  a,  b ,  and  c  is  conjectured.  Until  then,  if  a  conjecture  has 
transitions  on  an  action  other  than  a,  b,  and  c,  a  negative  sample  with  a  single 
transition  on  that  action  is  returned.  Otherwise,  the  tree  P  in  Fig.  5.3  is  returned 
as  the  positive  sample.  Note  that  P  has  an  execution  mapping  to  H\ .  It  is  easy  to 
see  that  such  samples  can  always  be  returned  until  the  learner  conjectures  H*. 

Once  H*  is  conjectured,  the  teacher  returns  Na  in  the  figure  to  make  sure  that 
every  future  conjecture  has  at  least  2  states.  If  a  future  conjecture  has  a  transition  on 
an  action  other  than  a,  b,  and  c,  the  teacher  can  similarly  return  a  negative  sample 
as  above.  If  not,  it  returns  P  or  Nb  in  the  figure,  if  possible.  Otherwise,  the  teacher 
returns  a  new  negative  sample  as  follows. 

Let  si  and  S2  be  the  2  states  of  the  current  conjecture  H .  Let  A„,  Alb,  and  A* 
be  the  sets  of  distributions  of  the  transitions  outgoing  from  s;,  i  G  {1,  2},  on  actions 
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a,  b  and  c,  respectively.  As  Na  ^  H ,  we  have  that  A*  ^  0  and  for  every  pa  G  A*, 
ha(si)  <  1  and  hence,  /za(s2)  >  0.  Similarly,  as  Nb  ^  H  and  P  ^  H,  we  have  that 
A£  7^  0  and  for  every  |ij  6  Aj,  /i&(s2)  >  0  and  every  S;  G  Supp(nb) ,  A*  7^  0.  That 
is,  every  transition  on  action  b  from  Si  has  non-zero  probability  of  going  to  s2  and 
every  state  in  its  support  has  a  transition  on  c.  The  teacher  then  returns  Nj?’1  in  the 
figure,  where  f3  =  pa(s2)  for  some  pa  G  A^  and  7  =  pc(s2)  for  some  pc  G  A;?.  Note 
that  Af ,7  has  an  execution  mapping  to  H . 

It  remains  to  show  that  there  always  exists  a  consistent  LPTS  of  2-states  for  the 
above  adversarial  strategy.  We  will  show  that  H\  in  the  figure  is  such  a  consistent 
LPTS.  We  have  seen  earlier  in  the  chapter  that  H\  simulates  P  and  does  not  simulate 
either  Na  or  W.  Moreover,  if  we  choose  A  G  (0 ,/3m*n),  where  /3min  is  the  minimum 
value  of  /3  in  all  Af ,7  samples  returned  so  far,  then  H\  does  not  simulate  the  current 
Af ,7  either.  Such  a  A  can  always  be  chosen  as  /3mm  >  0. 

We  conclude  that  there  is  no  converging  learning  algorithm  under  both  Condi¬ 
tion  1  on  the  teacher  and  Condition  2.  □ 


Despite  the  above  negative  result,  we  obtain  a  semi-algorithm  for  the  learning 
problem  under  both  the  conditions,  by  using  stochastic  state-space  partitioning  (Sec¬ 
tion  5.2.2)  for  FlNDCONSISTENT  in  every  iteration  of  the  active  learning  framework. 
That  is,  if  the  learner  converges,  it  is  guaranteed  to  learn  the  target  with  the  least 
number  of  states.  Correctness  is  immediate  from  Theorem  13. 
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5.4  Learning  Assumptions  for 
Compositional  Reasoning 

We  will  now  describe  how  the  active  learning  framework  LearnLPTS  can  be  used 
for  learning  a  sufficient  intermediate  assumption  A  in  the  rule  ASym  mentioned  in 
Section  5.1.  We  start  with  the  algorithms  for  the  teacher  and  the  learner  and  then 
briefly  describe  the  complexity  guarantees. 

Teacher 

The  teacher  simply  performs  two  conformance  checks  corresponding  to  the  two 
premises  of  the  rule.  If  a  conjecture  A  satisfies  both  the  premises,  the  teacher  re¬ 
turns  yes.  In  this  case,  the  conclusion  holds  as  well,  given  that  ASym  is  sound 
(Lemma  12).  If  one  of  the  premises  fails,  the  teacher  generates  a  new  sample  with 
an  execution  mapping ,  using  the  counterexample  generation  algorithm  described  in 
Section  4.5.  Thus,  the  teacher  satisfies  Condition  1.  If  premise  2  fails,  a  positive 
sample  is  returned  to  the  learner.  If  premise  1  fails,  the  obtained  counterexample 
C  is  first  projected  onto  A,  using  the  one-to-one  correspondence  from  C  to  L\  ||  A 
given  by  the  execution  mapping,  and  the  projection  is  returned  as  a  negative  sample 
(see  Section  6.3  for  more  details  on  projection). 

Learner 

The  learner  uses  the  state-space  (stochastic)  partitioning  techniques  described  in 
Section  5.2  for  inferring  a  new  conjecture  for  the  assumption  A  whenever  a  new 


171 


sample  is  returned  by  the  teacher.  As  every  positive  sample  has  an  execution  mapping 
to  L2,  the  learning  target  is  L2  from  the  learner’s  perspective.  This  works  because 
if  the  system  {L\  ||  L2)  conforms  to  P,  then  L2  is  clearly  an  assumption  satisfying 
the  premises.  However,  in  practice,  we  expect  the  algorithm  to  converge  to  a  smaller 
assumption  that  also  satisfies  the  premises. 


If  the  system  conforms  to  the  specification  P,  i.e. ,  if  the  conclusion  of  ASym 
is  actually  true,  then  the  learner  is  guaranteed  to  converge,  provided  it  uses  the 
state-space  partitioning  technique  (Lemma  26).  However,  if  it  uses  stochastic  state- 
space  partitions  instead,  convergence  is  not  guaranteed  as  we  saw  in  Section  5.3. 
Nevertheless,  using  stochastic  partitions  leads  to  a  semi-algorithm  for  the  problem 
of  learning  an  intermediate  assumption  of  the  least-size. 


If  the  system  does  not  conform  to  P,  however,  there  is  no  assumption  satisfying 
both  the  premises  of  ASym  (due  to  soundness  of  the  rule).  In  this  case,  we  are 
also  interested  in  computing  a  counterexample  to  Li  ||  L2  A  P.  For  this  purpose, 
the  learner  performs  a  spuriousness  check  on  the  samples  returned  by  the  teacher, 
similar  to  the  CEGAR  approach  [38].  We  restrict  the  spuriousness  check  to  negative 
samples  following  previous  approaches  [99].  In  our  case,  the  learner  simply  checks 
N  <  L2  for  a  negative  sample  N.  If  the  check  succeeds,  then  a  counterexample  can 
be  constructed  from  the  failure  of  L\  ||  N  P  P.  Otherwise,  the  learning  framework 
moves  on  to  the  next  iteration.  A  slightly  more  involved,  but  practical,  way  for 
detecting  spuriousness  of  a  negative  sample  is  described  in  the  next  chapter. 
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Time  Complexity  Analysis 


Let  us  now  analyze  the  time  complexity  of  assume-guarantee  reasoning  when  state- 
space  partitions  are  used  by  the  learner.  Note  that,  as  described  in  Section  4.4,  the 
time  complexity  of  checking  simulation  conformance  is  polynomial  in  the  sizes  of  the 
two  LPTSes.  So,  the  time  complexity  of  monolithic  reasoning  to  determine  Li  | 
L2  ^  P  is  0(poly(\Ll\  ■  \L2\,  |P|)),  where  \L\  denotes  maxdS^I,  \tl\),  the  maximum 
of  the  number  of  states  and  the  number  of  transitions  of  L. 

Let  d  =  |r2|  and  b  be  the  maximum  size  of  the  support  of  a  distribution  in  L2. 
Given  a  state  of  a  candidate  assumption  of  size  k  and  a  transition  of  L2,  there  can  be 
at  most  /cft-many  corresponding  outgoing  transitions  (taking  non-determinism  into 
account)  from  that  state.  For  k  states  and  d  distributions,  this  gives  an  upper  bound 
of  dkb+l .  Therefore,  there  are  2dkb+1  different  possible  candidates  of  size  k  to  consider. 
If  m  is  the  number  of  states  in  the  final  assumption  output  by  the  algorithm,  the 
total  number  of  iterations  of  the  learning  algorithm  is  then  given  by  0(2dmb+1).  Note 
that  m  =  0(1^1),  i.e.,  m  is  upper  bounded  by  the  number  of  states  in  L2. 

In  each  iteration,  in  the  worst-case,  the  learning  algorithm  enumerates  all  the 
candidate  assumptions  of  the  current  size  k  and  performs  simulation  checks  with  all 
the  negative  samples.  Each  of  these  checks  has  a  time  complexity  of  0(poly(\A\,  |A/"|, 

I N | max)),  where  A  is  the  final  assumption,  J\f  is  the  final  set  of  negative  samples 
and  \N\max  is  the  largest  value  of  \N\,  for  any  N  £  A f.  Thus,  the  total  worst-case 
time  complexity  of  the  learning  algorithm  for  computing  the  final  assumption  is 
0(poly(\A\,  \JV\,  | N | max )  •  2dmb+1).  Furthermore,  the  time  complexity  of  checking  the 
two  premises  of  ASym  (by  the  teacher)  is  0(poly(\Li\  ■  |A|,  |P|)  +  poly(\L2\,  |A|))  in 
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every  iteration.  We  observe  that,  if  the  final  assumption  is  small  (i.e.,  |A|  -C  \L2\) 
in  practice,  this  approach  can  be  better  than  monolithic  reasoning.  Moreover,  when 
A  is  small,  we  also  expect  memory  savings  in  practice.  In  other  cases,  however,  we 
would  need  better  algorithms  to  address  the  problem. 


5.5  Related  Work 

Learning  for  automating  compositional  reasoning  of  probabilistic  systems  has  been 
proposed  before  [52]  in  the  context  of  checking  probabilistic  reachability  properties, 
which  are  refuted  by  sets  of  trace  counterexamples.  The  approach  uses  a  variant  of 
L*  [14],  a  learning  algorithm  for  DFAs,  to  automatically  learn  deterministic  assump¬ 
tions,  following  previous  work  in  the  non-probabilistic  setting  [99].  The  approach 
uses  a  sound  but  incomplete  rule,  and  therefore,  it  is  not  guaranteed  to  terminate 
(completeness  is  necessary  for  termination).  A  complete  rule  for  such  properties  re¬ 
stricted  to  systems  without  non-determinism  has  been  considered  recently  [  |.  It 

uses  learning  with  probabilistic  trace  inclusion  as  the  conformance  relation  which  is 
undecidable.  Also,  the  learning  algorithm  is  not  guaranteed  to  terminate.  In  con¬ 
trast,  we  use  simulation  conformance  which  is  decidable  in  polynomial  time  and  leads 
to  a  sound  and  complete  rule  (Theorem  12).  We  are  also  able  to  guarantee  termina¬ 
tion  for  the  algorithm  proposed  in  Section  5.4  when  using  state-space  partitions  to 
infer  a  consistent  LPTS. 

Our  work  draws  inspiration  from  a  previous  work  [67]  that  automates  assumption 
generation  by  using  an  algorithm  for  learning  the  minimal  separating  automaton  from 
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positive  and  negative  trace  counterexamples.  The  counterexamples  are  provided  via 
model  checking  in  an  assume-guarantee  framework.  Similar  to  our  work,  they  use 
a  partitioning  approach ,  where  the  goal  is  to  find  a  folding  of  the  counterexamples 
into  the  learnt  model.  A  different  approach  has  been  proposed  to  find  the  separating 
automaton  based  on  L*  which  makes  use  of  membership  queries,  in  addition  to 
equivalence  queries  [33].  All  these  works  were  done  in  the  context  of  non- probabilistic 
reasoning  under  trace  semantics  and  thus,  are  different  from  our  setting. 

Learning  a  minimum-state  automaton  from  positive  and  negative  samples  is  a 
well  studied  problem  [15,  58,  97]  that  is  known  to  be  hard  [61].  Algorithms  have 
also  been  proposed  for  samples  with  stochastic  information,  i.e. ,  the  probability  of 
acceptance  of  a  trace  or  a  tree  [27,  28],  learning  stochastic  finite  (tree)  automata.  As 
also  previously  said,  we  cannot  immediately  borrow  existing  results  from  the  above 
automata-theoretic  approaches. 

LPTSes  are  related  to  probabilistic  automata  (PA)  [100].  Algorithms  to  learn 
PAs  have  only  been  proposed  in  restricted  settings  of  stronger  assumptions  on  a 
teacher  [104]  or  approximate  learning  [11,  87].  Algorithms  to  learn  a  multiplicity 
automaton,  which  generalizes  a  PA  by  replacing  the  probabilities  with  arbitrary  ra¬ 
tional,  have  also  been  proposed  [21].  Adapting  these  to  solve  verification  problems 
involving  probabilistic  transition  systems  is  difficult  and  results  in  non-terminating 
algorithms  [11].  On  the  other  hand,  we  show  in  Section  5.4  that  one  can  readily 
apply  the  algorithms  we  propose  to  infer  intermediate  assumptions  in  an  automated 
assume-guarantee  style  framework  for  the  verification  of  strong  simulation  confor¬ 
mance  between  LPTSes.  This  yields  the  first  complete  and  fully  automated  learning 
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framework  for  compositional  verification  of  probabilistic  systems.  Moreover,  one  can 
extend  this  framework  to  check  logical  properties,  such  as  the  fragment  weakly  safe 
PCTL  [29],  which  are  preserved  by  the  conformance  and  also  have  tree  counterex¬ 
amples. 

5.6  Conclusion 

We  have  presented  algorithms  and  decidability  results  for  the  problem  of  active 
learning  for  LPTSes  from  stochastic  tree  samples,  using  traditional  and  stochastic 
state-space  partitioning.  We  have  also  described  the  application  of  the  algorithms 
to  automating  the  discovery  of  assumptions  for  the  compositional  verification  of 
LPTSes. 

In  the  future,  it  would  be  interesting  to  investigate  further  conditions  on  the 
teacher  that  will  make  the  active  learning  problem  with  stochastic  partitions  decid¬ 
able.  The  learning  algorithms  presented  here  are  quite  general  and  not  restricted 
to  compositional  verification.  So,  another  interesting  future  direction  is  to  investi¬ 
gate  new  applications  of  our  algorithms  that  may  be  in  domains  outside  automatic 
verification. 

The  algorithms  presented  in  this  chapter  are  published  as  part  of  the  proceedings 
of  LICS  2012  [79]. 
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Chapter  6 


Abstraction  Refinement  for 
Simulation  Conformance 

6.1  Introduction 

In  previous  chapters,  we  have  described  the  problem  of  state-space  explosion  for 
checking  simulation  conformance  of  a  multi-component  Labeled  Probabilistic  Transi¬ 
tion  System  (LPTS)  against  a  specification  LPTS.  We  have  also  described  a  frame¬ 
work  for  compositional  reasoning  in  Chapter  5,  using  an  assume- guarantee  paradigm. 
To  recall,  the  assume-guarantee  reasoning  we  are  interested  in  is  captured  by  the  fol¬ 
lowing  inference  rule  for  the  case  of  two  components: 


1  :  U 


APP  2:  L2  PA 
-  (ASym) 

L\  |  L-2  P  P 
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In  the  last  chapter,  we  have  seen  iterative  algorithms  for  learning  a  suitable 
intermediate  assumption  A  that  satisfies  the  premises  of  the  above  rule,  utilizing 
counterexamples  to  the  premises  obtained  from  previous  conjectures  for  A.  In  this 
chapter,  we  describe  an  alternative  approach  based  on  automatic  abstraction  re¬ 
finement  [38].  In  this  approach,  the  assumption  A  is  maintained  as  a  conservative 
abstraction  of  L2,  i.e. ,  an  LPTS  that  simulates  L2  (and  hence,  premise  2  holds  by 
construction),  and  is  iteratively  refined  based  on  tree  counterexamples  obtained  from 
checking  premise  1.  Moreover,  we  use  a  state-space  partitioning  technique,  similar  to 
the  one  described  in  Section  5.2.1,  for  obtaining  such  an  abstraction  from  L2.  This 
ensures  that  the  iterative  process  is  guaranteed  to  terminate,  with  the  number  of 
iterations  bounded  by  the  number  of  states  of  L2. 

We  first  describe  an  automatic  abstraction  refinement  based  algorithm  for  check¬ 
ing  simulation  conformance  between  two  LPTSes  in  Section  6.2.  This  is  based  on  the 
well-known  Counter  Example  Guided  Abstraction  Refinement  (CEGAR)  approach  for 
non-probabilistic  systems  [38].  We  will  then  describe  how  to  adapt  CEGAR  to  the 
assume-guarantee  setting,  to  obtain  our  algorithm  Assume- Guarantee  Abstraction 
Refinement  (AGAR),  in  Section  6.3.  When  the  system  is  composed  of  more  than  2 
components,  i.e.,  when  L2  itself  is  composed  of  multiple  components,  we  can  apply 
assume-guarantee  reasoning  recursively  for  the  second  premise  (Lo  A  A).  We  extend 
the  rule  ASym  for  the  case  of  n  >  2  components  and  describe  how  AGAR  can  be 
naturally  extended  to  the  new  rule.  We  also  briefly  describe  how  AGAR  can  further 
be  applied  to  the  case  where  the  required  specification  is  given  as  a  property  in  a 
logic  preserved  by  strong  simulation.  We  implemented  the  algorithms  for  counterex- 
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ample  generation  (described  in  Section  4.5)  and  for  AGAR  in  Java  using  the  Yices 
SMT  solver  [46]  and  show  experimentally  in  Section  6.4  that  AGAR  can  achieve 
significantly  better  performance  than  monolithic  conformance  checking. 

Related  Work.  CEGAR  algorithms  for  probabilistic  systems  have  been  proposed 
earlier  in  the  context  of  probabilistic  reachability  [72]  and  safe-PCTL  [29],  The 
CEGAR  approach  we  describe  in  Section  6.2  is  an  adaptation  of  the  latter.  However, 
our  main  interest  is  in  the  compositional  setting  (Section  6.3). 


6.2  CEGAR  for  Checking  Strong  Simulation 

Assume  that  we  are  interested  in  checking  whether  L  P  P  holds  for  2  LPTSes 
L  =  (Sl,  s°L,aL,TL)  and  P  =  ( Sp,  s°P,ap,Tp ).  We  describe  an  algorithm  based  on 
automatic  abstraction  refinement  to  infer  an  LPTS  A  that  simulates  L  (in  which 
case,  we  refer  to  A  as  an  abstraction  of  L)  while  conforming  to  P,  i.e. ,  L  A  A  A  P . 
By  Lemma  13,  it  will  then  follow  that  L  A  P.  The  algorithm  is  based  on  the  well- 
known  Counter  Example  Guided  Abstraction  Refinement  (CEGAR)  approach  [38]. 
Such  a  CEGAR  approach  can  be  useful  when  checking  L  A  P  directly  is  expensive. 
Moreover,  we  will  see  how  CEGAR  can  be  adapted  to  the  assume-guarantee  setting 
in  Section  6.3. 

Fig.  6.1  shows  the  pseudo-code  of  the  CEGAR  algorithm  for  simulation  confor¬ 
mance  between  LPTSes.  The  algorithm  maintains  an  abstraction  A  of  L  as  the 
quotient  of  a  state-space  partition  of  L.  The  partition  and  the  quotient  construction 
are  as  described  in  Section  5.2.1.  The  algorithm  initializes  a  variable  n,  denoting  a 
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1 

2 

3 

4 

5 
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CEGAR(L  =  (SL,s°L,aL,TL),P  =  ( SP,s0p,ap,TP }) 
II  <r-  the  coarsest  partition  of  Sl 
A  <r~  L/H 
while  true  do 

(res,  C,  M )  «-  CheckSim(A,  P) 
if  res  =  yes  then 
[  return  (yes,  — ) 


7  (spurious,  II,  R)  <-  AnalyzeAndRefine(C,  L,  II,  A,  M) 

8  if  spurious  then 

9  |_  A  <—  L/n 


/ /  see  text 


10 

11 


else 

[^  return  (no,  C ) 


Figure  6.1:  CEGAR  algorithm  for  checking  L  A  P- 


state-space  partition  of  Sl,  to  the  coarsest  partition  where  there  is  only  one  equiv¬ 
alence  class  which  contains  all  the  states  (line  1).  The  algorithm  then  iteratively 
refines  fl  (and  hence,  refines  A)  based  on  the  counterexamples  obtained  from  the 
simulation  check  A  =  L/U  ^  P  (denoted  by  CheckSim  on  line  4).  If  A  ^  P  can  be 
shown,  we  can  conclude  L  <  P  (lines  5-6).  Otherwise,  we  obtain  a  counterexample 
C  to  L  ^  P  which  we  need  to  check  for  feasibility  in  L  (see  below  for  details).  For 
this  purpose,  we  use  the  routine  AnalyzeAndRefine  (line  7),  which  we  explain 
below.  We  also  describe  how  the  routine  refines  the  partition  II  in  case  C  is  spurious, 
which  can  then  be  used  to  update  the  abstraction  A  (line  9).  However,  if  C  is  feasible 
in  L ,  then  we  have  found  a  counterexample  to  L  A  P  which  we  return  on  line  11. 
Our  counterexample  analysis  explained  below  is  an  adaptation  of  an  existing  one 
for  counterexamples  which  are  arbitrary  substructures  of  A  [29];  our  stochastic  tree 
counterexamples  are  not  necessarily  sub-structures  of  A.1 

1A  sub-structure  of  an  LPTS  L,  at  a  high  level,  is  an  LPTS  that  can  be  obtained  from  L  by 
removing  some  transitions.  A  formal  presentation  can  be  found  elsewhere  [29]. 


180 


Spuriousness  Check  and  Refinement  (AnalyzeAndRefine).  Let  II  be  a 

partition  and  A  =  L/Ii  be  such  that  A  -£■  P  and  let  C  =  (Sc,  s0c,ac,Tc)  be  a 
counterexample  with  an  execution  mapping  M  from  Sc  to  Sa ■  Our  goal  is  now  to 
check  whether  C  is  feasible  in  L ,  i.e.,  whether  C  R  L.  If  it  is  feasible,  C  is  a  real 
counterexample  as  we  already  know  C  ^  P.  If  it  is  not  feasible,  we  need  to  refine  A  by 
refining  the  partition  II.  Fig.  6.2  shows  the  pseudo-code  for  AnalyzeAndRefine, 
which  is  essentially  an  instrumented  version  of  the  algorithm  for  checking  simulation 
conformance  for  trees  from  Section  4.4  (see  ComputeSimTree  in  Fig.  4.11).  With 
the  goal  of  partition  refinement,  we  compute  the  coarsest  strong  simulation  between 
C  and  L  contained  in  Rm  =  {(si,S2)  |  si  G  Sc,S2  6  Sl,M(s i)  =  [s2]ri},  as  opposed 
to  Sc  x  Sl  (line  1).  In  words,  Rm  relates  a  state  si  of  Sc  to  all  states  of  Sl  in  the 
equivalence  class  M(s\).  The  analysis  works  as  follows. 

Intuitively,  the  algorithm  does  a  bottom-up  traversal  of  C  and  for  each  transition 
Si  — >  Hi,  checks  whether  it  is  simulated  by  a  transition  from  a  state  in  R(s\),  where 
R  is  the  current  value  of  the  binary  relation  maintained  by  the  algorithm.  After  every 
iteration  of  the  for-loop  beginning  at  line  4,  the  algorithm  checks  for  the  following 
two  cases  and  refines  the  partition  or  continues  with  the  next  iteration.  Let  R0id  be 
the  value  of  R  at  the  beginning  of  every  iteration  of  this  for-loop  (see  lines  2  and  20). 

1.  R(s\)  =  0:  There  are  two  possible  reasons  for  this  case.  One  is  that  no 
state  in  Rm(s i)  simulates  all  the  outgoing  transitions  of  si,  in  which  case  the 
equivalence  class  M(s\)  is  a  candidate  for  refinement.  The  other  reason  is 
that  R  does  not  relate  the  states  in  Supp(jii)  sufficiently  enough  to  the  states 
in  L  for  the  transition  to  be  simulated  by  a  state  in  Rm (s i  )  •  In  this  case, 
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the  equivalence  classes  corresponding  to  the  states  in  Supp(p{)  are  candidates 
for  refinement.  As  shown  on  lines  14-15,  we  refine  every  equivalence  class  in 
(M(s)  |  s  G  {si}  U  Supp(fjii)}  by  splitting  M(s)  into  R0m(s),  which  is  a  subset 
of  M(s)  by  construction,  and  the  rest. 

2.  R(si)  7^  0,  but  M(si)  =  [s°]n  and  s°L  G  R0id(si)  \  R(s i):  In  this  case,  M(si)  is 
the  initial  state  of  the  abstraction  A  but  si  is  no  longer  related  to  the  initial 
state  s°L  of  L.  Here,  the  equivalence  class  M(s i)  is  a  candidate  for  refinement 
and  we  split  it  into  R0id(s l)  \  R(s l)  and  the  rest  (line  18). 

The  following  lemma  shows  that  the  above  mentioned  refinement  strategy  is  guar¬ 
anteed  to  result  in  a  strictly  finer  partition  and  hence,  refine  the  abstraction  A. 
Lemma  27.  If  AnalyzeAndRefine(C',  L,  n,  A,  M)  returns  (yes,  n',  — ),  then  A' 
is  a  strictly  finer  partition  than  n,  i.e.,  n'  <  n. 

Proof.  It  suffices  to  show  that  on  lines  15  and  18,  at  least  one  equivalence  class  gets 
split  into  two  non-empty  subsets. 

Consider  the  first  case  where  R(s i)  =  0  for  some  si  G  Sc  on  line  13  of  the 
pseudo-code  in  Fig.  6.2.  If  R0id(s i)  C  RM(s i),  then  we  are  done.  Otherwise,  it  must 
be  the  case  that  R0id(s )  C  RM(s )  for  some  s  G  Supp(pi).  This  is  because,  si  and 
pi  are  related  to  A  by  the  execution  mapping  M  and  the  corresponding  distribution 
in  A  is  obtained  by  lifting  a  distribution  of  L.  So,  if  R0id(s)  =  Rm(s )  for  every 
s  G  Supp(pi),  one  can  then  show  that  R(s\ )  could  not  have  been  empty. 

In  the  second  case,  s°L  G  R0id(s i)  \  R(si )  and  0  7^  R(s  1)  C  R0id(s  1).  Clearly, 
splitting  M(si )  results  in  two  non-empty  subsets.  □ 
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AnalyzeAndRefine(C,  L,  n,  A  =  L/ n,  M  :  Sc  ->  Sa) 

//  BEGIN  INSTRUMENTATION 

1  R  {(si,  S2)  I  Si  £  Sc,  S2  £  Sl,M(si)  =  [s2]n} 

//  relate  Si  with  all  states  in  the  equivalence  class  M(s\) 

2  Sold  £ -  i? 

//  END  INSTRUMENTATION 


3 

4 

5 

6 

7 

8 
9 

10 


for  every  non-leaf  Si  €  Sc  in  a  bottom-up  traversal  of  C  do 
for  every  si  A  p\  do 

for  every  S2  £  R(si)  do 
sim  t—  false 

for  every  s 2  A  P2  do 
if  Pi  Efl  P2  then 
sim  <—  true 
break 


11 

12 


if  sim  =  false  then 

|_  R  R  \  {(si,  S2)} 


13 

14 

15 

16 

17 

18 
19 


//  BEGIN  INSTRUMENTATION 

if  R(si)  =  0  then 

for  every  s  £  {si}  U  Supp(/ii)  do 
j  refine  II  by  splitting  M(s)  into  R0id(s)  and  the  rest 

return  (yes,  EE,  — ) 

else  if  M(s\)  =  [s°]n  and  R(si)  then 

refine  II  by  splitting  M(s  1)  into  R0id(s  1)  \  R(s  1)  and  the  rest 
return  (yes,  II,  — ) 


20 


Rold  t—  R 

//  END  INSTRUMENTATION 


21 


return  (no,  — ,  R) 


Figure  6.2:  Counterexample  analysis  and  partition  refinement,  obtained  by  instrumenting  Com- 
puteSimTree  in  Fig.  4.11. 
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If  neither  of  the  above  mentioned  cases  for  refinement  is  encountered  before  reach¬ 
ing  the  end  (line  21),  clearly,  R(sq)  0  and  s°L  G  R{sq)  and  hence,  the  final  value 
of  R  is  a  strong  simulation  between  C  and  L  that  relates  the  initial  states.  In  other 
words,  C  is  a  real  counterexample. 


6.3  Assume- Guarantee  Abstraction  Refinement 

We  will  now  describe  how  CEGAR  can  be  adapted  to  the  assume-guarantee  setting, 
which  we  call  Assume- Guarantee  Abstraction  Refinement  (AGAR).  The  notable  dif¬ 
ference  with  CEGAR  is  that  the  counterexample  analysis  is  performed  in  an  assume 
guarantee  style:  a  counterexample  obtained  from  checking  one  component  (together 
with  an  abstraction  of  the  environment,  i.e.,  the  other  components)  is  used  to  refine 
the  abstraction  of  a  different  component. 

Fig.  6.3  shows  the  pseudo-code  of  our  AGAR  algorithm.  Given  LPTSes  Ll5  L2 
and  P,  the  goal  is  to  check  Li  ||  L2  A  P  in  an  assume-guarantee  style,  using  the 
rule  ASym.  The  basic  idea  is  to  maintain  A  in  the  rule  as  an  abstraction  of  L2l  i.e., 
the  second  premise  (. L2  A  A)  holds  for  free  throughout,  and  to  check  only  the  first 
premise  (L\  ||  A  P  P)  for  the  abstraction  A  maintained  by  the  algorithm.  As  in 
CEGAR,  we  restrict  A  to  the  quotient  of  a  state-space  partition  of  S2,  the  states  of 
L2.  If  the  first  premise  holds  for  A,  then  L\  \\  L2  P  P  also  holds,  by  the  soundness 
of  the  rule.  Otherwise,  the  obtained  counterexample  C  is  analyzed  to  see  whether 
it  indicates  a  real  error  or  is  spurious,  in  which  case  A  is  refined.  The  spuriousness 
analysis  and  refinement  are  compositional,  as  explained  below. 
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AGAR(Li,L2,P) 

II  <r-  the  coarsest  partition  of  S2 

A  -f-  l2/ n 

while  true  do 

(res,  C,  M)  <-  CheckSim(Ai  ||  A,P) 
if  res  =  yes  then 
j_  return  (yes,  — ) 

(C[a,M[a)  4-  Project(C,  A,  M) 

(spwicras,n,  i?)  «-  AnalyzeAndRefine(C'[.j4,  L2,  n,  A,  M[a) 
if  spurious  then 
j  A  <—  L2/Tl 

else 

j  return  ( no,C ) 


Figure  6.3:  AGAR  algorithm  for  checking  L\  ||  L2  A  P  in  an  assume-guarantee  style. 

Analysis  and  Refinement.  In  AGAR,  the  counterexample  analysis  is  performed 
compositionally,  using  the  projections  of  C  onto  L\  and  A  obtained  as  follows.  As 
in  the  case  of  CEGAR,  we  make  use  of  an  execution  mapping  M  from  Sc  to  5Vn|u- 
From  Definition  12,  we  know  that  every  state  of  Li  ||  A  is  a  pair  of  states  of  L 1  and 
A,  and  every  distribution  of  a  transition  in  Li  ||  A  is  the  product  of  two  distributions, 
one  each  from  Li  and  A.  So,  we  can  utilize  the  one-to-one  correspondence  from  the 
states  and  distributions  of  C  to  those  of  Li  ||  A,  given  by  M ,  and  pick  the  respective 
components  of  the  state  pairs  and  products  of  distributions  to  obtain  what  we  call 
projections  of  C  onto  Li  and  A.  We  denote  these  projections  by  C[.l1  and  C(a, 
respectively.  Note  that  there  is  a  natural  execution  mapping  from  C[a  to  A,  which 
we  denote  by  M(a  (line  5).  We  will  then  use  AnalyzeAndRefine,  described  in 
Section  6.2,  to  check  whether  C(a  is  simulated  by  L2  or  if  it  is  spurious  and  refine  the 
partition  and  the  abstraction  A  (line  6).  If  AnalyzeAndRefine  returns  no,  i.e. ,  it 
concludes  that  C(a  is  not  spurious,  we  can  conclude  that  C  is  a  counterexample  to 


185 


L\  ||  Lo  S  P ,  as  the  following  lemma  shows. 

Lemma  28.  If  AnalyzeAndRefine  returns  no  at  line  6  of  Agar,  then  then  C , 
found  on  line  f,  is  a  counterexample  to  Li  ||  L2  A  P. 

Proof.  For  an  LPTS  L  =  ( Sl ,  s°L ,  aL,  tl)  and  alphabet  (3  such  that  oil  C  (3,  we  write 
L13  to  denote  the  LPTS  (Sl,  s°l,  (3,  tl).  Let  Si  and  a*  be  the  set  of  states  and  alphabet 
for  the  LPTS  U. 

If  AnalyzeAndRefine  returns  no,  then  C[a  S  A  holds.  So,  (C|,a)"2  S  L2 
also  holds.  Moreover,  we  know  that  the  projection  C[l1  satisfies  (C|,l1)"1  L\.  To¬ 

gether,  it  follows  that  (CtiJ"1  ||  (C|.u)“2  ■<  Li  ||  L2  (Lemmas  19  and  13).  Moreover, 
the  projections  can  also  be  shown  to  satisfy  C  (CIlJ"1  ||  (CU)a2-  As  C  -ff  P  is 
already  known,  it  follows  from  Lemma  13  that  C  is  a  real  counterexample.  □ 

The  following  is  immediate  from  Theorem  12  and  Lemmas  27  and  28. 

Theorem  17  (Correctness  and  Termination).  Agar  is  guaranteed  to  terminate 
within  1 62 1  —  1  iterations,  the  conformance  L\  ||  L2  Si  P  holds  iff  it  returns  on 
line  1 1  and  fails  to  hold  iff  it  returns  on  line  1 0. 

In  practice,  we  expect  AGAR  to  take  less  than  \S2\  —  1  iterations,  terminating 
with  an  assumption  smaller  than  L2.  AGAR  will  terminate  as  soon  as  it  finds  an 
assumption  that  satisfies  the  premises  or  that  helps  exhibit  a  real  counterexample. 
Note  also  that,  although  AGAR  uses  an  explicit  representation  for  the  individual 
components,  it  never  builds  L \  ||  L2  directly  (except  in  the  worst-case)  keeping  the 
cost  of  verification  low. 

For  example,  Fig.  6.4  shows  a  specification  LPTS  for  the  2-component  input- 
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a,  s 


Figure  6.4:  A  specification  for  L\  ||  L2,  where  L\  and  L2  are  in  Fig.  4.16. 


s 


A 


Figure  6.5:  An  assumption  for  L\  ||  L2  in  Fig.  4.16  and  specification  P  in  Fig.  6.4. 

output  system  we  saw  in  Fig.  4.16.  For  this  example,  our  algorithm  AGAR  generates 
the  sufficient  assumption  A  shown  in  Fig.  6.5. 


6.3.1  Reasoning  with  n  >  2  Components 

So  far,  we  have  discussed  assume-guarantee  reasoning  in  the  context  of  two  compo¬ 
nents  L i  and  L2.  This  reasoning  can  be  generalized  to  n  >  2  components  using  the 
following  rule  ASym-N  (which  can  similarly  be  shown  to  be  sound  and  complete). 
This  rule  enables  us  to  overcome  the  intermediate  state-space  explosion  that  may  be 
associated  with  two-way  decompositions  when  the  subsystems  are  larger  than  the 
entire  system. 


l-.Ll\\Al<P  2  :  L2  ||  A2  ^  Ai 


n 

2—1 


Li  ±P 


^  •  Ln  ^  An—\ 

-  (ASym-N) 


187 


1 


AGAR-N  ((L1,...,Ln),An,P) 

P  <—  Ln 

2  if  An  A  e  then  //  An  =  e  holds  only  for  the  very  first  call 

3  P  P  ||  An 


4 

5 

6 
7 


if  n  =  1  then 

(res,  C,  M)  v-  CheckSim(P,  P) 
(CUn,MUj  e-  Project(C,  An, M) 
return  (res,  C,  A/) 


8 

9 

10 

11 

12 

13 


14 

15 

16 

17 


18 

19 

20 


//  compute  sufficient  assumption  A  for  (P i  ||  •  •  •  ||  P„_ i)  |j  P  ^  P 
II  ■(—  coarsest  partition  of  Sl 

A  ^  L/n 

while  true  do 

(res,  C,  M )  <-  AGAR-N((Li, . . . ,  L„_i),  A,  P) 
if  res  =  yes  then 
[  return  (yes,  — ,  — ) 

//  C  A  A  with  execution  mapping  M 

{spurious,  n,  R)  AnalyzeAndRefine(G,  L,  n,  A,  M ) 
if  spurious  then 
|_  A^L/U 

else 

//  P  is  a  strong  simulation  between  C  and  L 
//  but  we  also  need  an  execution  mapping 

(: Tl,Ml ) «-  ObtainCexWithMapping(G,  P,  P) 

(Tl  U„ ,  Ml  U  J  -f-  Pro.tect(Pl,  A„,  ML) 
return  (no ,  Tj,  Un ,  Mi  U„ ) 


Figure  6.6:  AGAR  algorithm  for  checking  Pi  ||  •  •  •  ||  Ln  ||  An  A  P  for  n  >  2. 


Fig.  6.6  shows  the  pseudo-code  for  AGAR-N,  the  adaptation  of  AGAR  for  the 
rule  ASym-N.  The  algorithm  takes  as  input  n  components  (Li, . . . ,  Ln)  and  an  en¬ 
vironmental  assumption  An,  which  abstracts  the  rest  of  the  components  (if  any), 
which  should  together  conform  to  a  specification  P.  Initially,  the  environmental  as¬ 
sumption  is  absent,  which  we  denote  by  letting  An  to  be  e.  AGAR.-N  then  tries  to 
compute  an  assumption  A,  in  the  sense  of  the  two-component  rule  ASym,  by  using 
the  two-way  decomposition  (L1  ||  •  •  •  ||  Ln_i)  ||  (Ln  ||  An)  P  P  as  follows. 
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It  first  composes  Ln  with  An,  to  obtain  an  LPTS  L  (lines  1-3).  If  there  was 
only  one  component  to  begin  with  (line  4),  it  performs  a  monolithic  simulation 
conformance  check  (denoted  by  CheckSim  on  line  5)  and  returns  the  result.  If 
a  counterexample  is  returned  on  line  5,  along  with  an  execution  mapping  M,  it  is 
projected  onto  the  assumption  An  (line  6)  so  that  the  caller  can  check  for  spuriousness 
and  refine  An.  On  the  other  hand,  if  n  >  1,  the  desired  assumption  A  is  computed 
by  means  of  abstraction  refinement  of  L  (lines  8-20),  similar  to  AGAR,  recursively 
invoking  AGAR-N  for  (Li, . . . ,  Ln_i)  with  the  current  abstraction  A  of  L  as  the 
environmental  assumption  (line  11). 


If  the  recursive  call  to  AGAR-N  returns  yes,  then  A  is  a  sufficient  assumption 
(line  13).  Otherwise,  we  obtain  a  counterexample  C  with  an  execution  mapping 
M  from  Sc  to  Sa •  The  goal  is  now  to  check  whether  C  is  spurious  and  refine  A 
in  case  it  is.  This  is  done  similar  to  AGAR  (line  14-16).  If  C  is  not  spurious, 
then  the  environmental  assumption  An  needs  to  be  checked  for  refinement.  But, 
we  cannot  simply  project  C  onto  An,  as  C  does  not  have  an  execution  mapping 
to  L.  However,  the  call  to  AnalyzeAndRefine  on  line  14  also  returns  a  strong 
simulation  R  between  C  and  L  that  relates  the  initial  states.  We  can  then  use  R  to 
obtain  a  new  tree  Tl  by  simulating  every  transition  in  C,  say  in  a  top-down  traversal 
of  C  (ObtainCexWithMapping  on  line  18).  We  can  show  that  C  Y  Tjj  y<  L  and 
hence,  we  can  use  Tl  instead  of  C.  As  Tl  is  essentially  an  unrolling  of  L,  we  can 
also  obtain  an  execution  mapping  to  L ,  say  Ml-  We  can  then  project  Tl  and  Ml 
onto  An  (line  19)  which  are  returned  to  the  caller  (line  20). 
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6.3.2  Compositional  Verification  of  Logical  Properties 

AGAR  can  be  further  applied  to  automate  assume-guarantee  verification  of  prop¬ 
erties  written  as  formulae  in  a  logic  that  is  preserved  by  strong  simulation,  such  as 
the  weak-safety  fragment  of  probabilistic  CTL  (PCTL)  [29]  which  also  admits  tree 
counterexamples.  The  modified  rule  ASym  is  both  sound  and  complete  for  this  logic 
(|=  denotes  property  satisfaction),  provided  a  a  Q  a  2  with  a  proof  similar  to  that  of 
Theorem  12. 


1  :  Lx  ||  A  |=  0  2  :  L2AA 

L\  ||  L*2  H  0 

The  intermediate  assumption  A  can  be  similarly  maintained  as  a  conservative 
abstraction  of  L2  and  iteratively  refined  based  on  the  tree  counterexamples  to  premise 
1,  using  the  same  procedures  as  before.  The  rule  can  be  generalized  to  reasoning 
about  n  >  2  components  as  described  above  and  also  to  richer  logics  with  more 
general  counterexamples  adapting  existing  CEGAR  approaches  [29]  to  AGAR.  We 
plan  to  further  investigate  this  direction  in  the  future. 

6.4  Implementation  and  Results 

Implementation.  We  implemented  the  algorithms  for  checking  strong  simulation 
and  generating  counterexamples,  which  we  described  in  Chapter  4,  as  well  as  AGAR 
and  AGAR-N  in  Java.  We  used  the  front-end  of  the  tool  PRISM  [82]  to  parse 
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the  models  of  the  components  described  in  PRISM’s  input  language  and  construct 
LPTSes  which  are  then  handled  by  our  implementation. 

As  mentioned  in  Chapter  4,  in  order  to  take  advantage  of  the  efficient  SMT  solvers 
that  exist  today,  we  used  the  SMT  encoding  given  by  ENCODESlM  in  Fig.  4.9  for 
checking  simulation  conformance.  It  follows  from  Lemma  15  that  the  constraints 
generated  by  the  SMT  encoding  are  satishable  iff  the  conformance  check  succeeds. 
However,  when  the  conformance  check  fails  to  hold,  there  is  no  direct  way  to  obtain 
a  tree  counterexample.  Note  that  in  this  case,  the  constraints  generated  by  the  SMT 
encoding  are  unsatishable.  So,  when  the  conformance  check  fails  to  hold,  say  between 
LPTSes  L\  and  L2,  we  obtain  an  unsatishable  subset  of  the  constraints,  by  utilizing 
the  unsat  core  extraction  facility  provided  by  the  Yices  SMT  solver.  From  this  subset 
of  constraints,  we  then  construct  a  sub-structure  of  L 1  and  check  the  conformance  of 
this  sub-structure  against  L2  using  the  Java  implementation.  This  sub-structure  is 
usually  much  smaller  than  Li  and  contains  only  the  information  necessary  to  expose 
the  counterexample. 

Results.  We  evaluated  our  algorithms  using  this  implementation  on  several  exam¬ 
ples  analyzed  in  previous  work  [52],  Some  of  these  examples  were  created  by  introduc¬ 
ing  probabilistic  failures  into  non-probabilistic  models  used  earlier  [99]  while  others 
were  adapted  from  PRISM  benchmarks  [82],  The  properties  used  previously  were 
about  probabilistic  reachability  and  we  created  our  own  specification  LPTSes  after 
developing  an  understanding  of  the  models.  The  models  in  all  the  examples  satisfy 
the  respective  specifications.  We  briefly  describe  the  models  and  the  specifications 
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below.2 


CSi  and  CSn  model  a  Client-Server  protocol  with  mutual  exclusion  having  prob¬ 
abilistic  failures  in  one  or  all  of  the  N  clients,  respectively.  The  specifications 
describe  the  probabilistic  failure  behavior  of  the  clients  while  hiding  some  of 
the  actions  as  is  typical  in  a  high  level  design  specification. 

MER  models  a  resource  arbiter  module  of  NASA’s  software  for  Mars  Exploration 
Rovers  which  grants  and  rescinds  shared  resources  for  several  users.  We  consid¬ 
ered  the  case  of  two  resources  with  varying  number  of  users  and  probabilistic 
failures  introduced  in  all  the  components.  As  in  the  above  example,  the  spec¬ 
ifications  describe  the  probabilistic  failure  behavior  of  the  users  while  hiding 
some  of  the  actions. 

SN  models  a  wireless  Sensor  Network  of  one  or  more  sensors  sending  data  and 
messages  to  a  process  via  a  channel  with  a  bounded  buffer  having  probabilistic 
behavior  in  the  components.  Creating  specification  LPTSes  for  this  example 
turned  out  to  be  more  difficult  than  the  above  examples,  and  we  obtained  them 
by  observing  the  system’s  runs  and  by  manual  abstraction. 

Table  6.1  shows  the  comparison  of  running  time  and  memory  consumption  among 
ASym,  ASym-N,  and  monolithic  (non-compositional)  conformance  checking.  Time 
is  in  seconds  and  Memory  is  in  megabytes.  Table  6.2  compares  the  sizes  of  various 
LPTSes  constructed  by  the  approaches.  \X\  stands  for  the  number  of  states  of  an 
LPTS  X.  L  stands  for  the  whole  system,  P  for  the  specification,  Lm  for  the  LPTS 

2 All  models  and  specifications  are  available  at 
http : //www. cs . emu . edu/~akomur av/project s/agar /AGAR. html. 
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Example 

ASym 

ASym-N 

Mono  | 

{par am) 

Time 

Mem 

Time 

Mem 

Time 

Mem 

CSi(5) 

7.2 

15.6 

74.0 

15.1 

0.2 

8.8 

CS i(6) 

11.6 

22.7 

810.7 

21.4 

0.5 

12.2 

C5i(7) 

37.7 

49.4 

out 

- 

0.8 

17.9 

CSn(2) 

0.7 

7.1 

2.4 

6.8 

0.1 

5.9 

CSN(  3) 

43.0 

63.0 

1.6k 

109.6 

14.8 

37.9 

CSN(  4) 

out 

- 

out 

- 

1.8fc 

667.5 

MER  (3) 

2.6 

19.7 

3.6 

14.6 

193.8 

458.5 

MER  (4) 

15.0 

53.9 

34.7 

37.8 

out 

- 

MER  (5) 

- 

out 1 

257.8 

65.5 

- 

out 1 

SN  (1) 

0.2 

6.2 

1.7 

8.5 

1.5 

27.7 

SN  (2) 

79.5 

112.9 

694.4 

171.7 

4.7fc 

1.3fc 

SN  (3) 

out 

- 

7.2  k 

528.8 

- 

out 

Table  6.1:  Time  and  Memory  consumption  for  AGAR  vs  monolithic  verification.  1  Mem-out  during 
model  construction. 


Example 

ASym 

ASym-N 

Mono 

(param) 

\Li\ 

M 

\Lm  I 

\Am\ 

\Lc\ 

\Lm  | 

\Am\ 

\L\ 

\P  1 

CS\  (5) 

36 

405 

182 

33 

36 

182 

34 

94 

16 

CSi(6) 

49 

1215 

324 

41 

49 

324 

40 

136 

19 

CSi(7) 

64 

3645 

538 

56 

64 

- 

- 

186 

22 

CSN(  2) 

25 

9 

51 

7 

9 

40 

25 

34 

15 

CSn(3 ) 

125 

16 

324 

12 

16 

372 

125 

184 

54 

CSjv(  4) 

625 

25 

- 

- 

25 

- 

- 

960 

189 

MER  (3) 

278 

1728 

706 

7 

278 

706 

7 

16fc 

12 

MER  (4) 

465 

21A; 

2k 

11 

465 

2k 

11 

120fc 

15 

MER  (5) 

700 

250A; 

- 

- 

700 

3.3k 

16 

841fc 

18 

SN  (1) 

43 

32 

43 

3 

126 

165 

6 

462 

18 

SN  (2) 

796 

32 

796 

3 

252 

1.4fc 

21 

7860 

54 

SN  (3) 

7545 

32 

- 

- 

378 

1.4A; 

21 

78  k 

162 

Table  6.2:  Sizes  of  various  LPTSes  constructed  for  AGAR  vs  monolithic  verification.  1  Mem-out 
during  model  construction. 


with  the  largest  number  of  states  built  by  composing  LPTSes  during  the  course  of 
AGAR,  Am  for  the  assumption  with  the  largest  number  of  states  during  the  execution 
and  Lc  for  the  component  with  the  largest  number  of  states  in  ASym-N.  We  also 
compared  \Lm\  with  \L\,  as  \Lm\  denotes  the  largest  LPTS  ever  built  by  AGAR.  Best 
figures,  among  ASym,  ASym-N  and  Mono,  for  Time,  Memory  and  LPTS  sizes,  are 
boldfaced.  All  the  results  were  obtained  on  a  Fedora-10  64-bit  machine  running  on 
an  Intel®  Core™2  Quad  CPU  of  2.83GHz  and  4GB  RAM.  We  imposed  a  2GB 
upper  bound  on  Java  heap  memory  and  a  2  hour  upper  bound  on  the  running  time. 
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We  observed  that  most  of  the  time  during  AGAR  was  spent  in  checking  the  premises 
and  an  insignificant  amount  was  spent  for  the  composition  and  the  refinement  steps. 
Also,  most  of  the  memory  was  consumed  by  Yices.  We  tried  several  orderings  of  the 
components  (the  L{'s  in  the  rules)  and  report  only  the  ones  giving  the  best  results. 

While  monolithic  checking  outperformed  AGAR  for  Client-Server ,  there  are  sig¬ 
nificant  time  and  memory  savings  for  MER  and  Sensor  Network  where  in  some  cases 
the  monolithic  approach  ran  out  of  resources  (time  or  memory).  One  possible  reason 
for  AGAR  performing  worse  for  Client-Server  is  that  \L\  is  much  smaller  than  \L{\ 
or  \L2\.  When  compared  to  using  ASym,  ASym-N  brings  further  memory  savings 
in  the  case  of  MER  and  also  time  savings  for  Sensor  Network  with  parameter  3 
which  could  not  finish  in  2  hours  when  used  with  ASym.  As  already  mentioned, 
these  models  were  analyzed  previously  with  an  assume-guarantee  framework  using 
learning  from  traces  [52].  Although  that  approach  uses  a  similar  assume-guarantee 
rule  (but  instantiated  to  check  probabilistic  reachability )  and  the  results  have  some 
similarity  (e.g.  Client-Server  is  similarly  not  handled  well  by  the  compositional  ap¬ 
proach),  we  can  not  directly  compare  it  with  AGAR  as  it  considers  a  different  class 
of  properties. 


6.5  Conclusion 

We  described  a  complete,  fully  automated  abstraction-refinement  approach  for  assume- 
guarantee  reasoning  of  strong  simulation  conformance  between  LPTSes.  The  ap¬ 
proach  uses  refinement  based  on  stochastic  tree  counterexamples  and  it  further  ap- 
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plies  to  checking  safe- PCTL  properties.  We  showed  experimentally  the  merits  of  the 
proposed  technique  in  comparison  to  monolithic  conformance  checking.  In  future, 
it  would  be  interesting  to  extend  the  approach  to  cases  where  the  assumption  A 
is  allowed  to  have  a  smaller  alphabet  than  that  of  the  component  it  abstracts  as 
this  can  potentially  lead  to  further  savings.  Strong  simulation  would  no  longer  work 
and  one  would  need  to  use  weak  simulation  [102],  whose  decidability  is  not  known 
yet  to  the  best  of  our  knowledge.  In  an  orthogonal  direction,  it  is  interesting  to 
explore  symbolic  implementations  of  our  algorithms,  for  increased  scalability.  An 
experimental  comparison  of  the  approach  presented  in  this  chapter  with  the  active 
learning  based  algorithms  from  the  previous  chapter  is  also  interesting.  However, 
this  requires  one  to  first  evaluate  the  algorithms  from  the  previous  chapter  from  a 
practical  perspective  and/or  investigate  practical  implementations  of  the  algorithms. 

The  results  presented  in  this  chapter  are  published  as  part  of  the  proceedings  of 
CAV  2012  [78], 
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Chapter  7 


Future  Work 


There  are  several  exciting  research  directions  one  can  extend  the  thesis  work  to  in 
the  future.  We  will  briefly  describe  some  of  them  below. 

Proof  Based  Abstraction  from  different  under-approximations.  As  we  have 
mentioned  right  in  Chapter  1,  a  central  theme  of  SAT/SMT-based  model  checking 
is  to  iteratively  solve  Bounded  Model  Checking  (BMC)  problems,  obtain  proofs  of 
bounded  safety,  and  try  to  generalize  the  proofs  to  invariants  of  the  entire  program. 
More  generally,  the  strategy  used  is  to  under-approximate  and  generalize,  in  an  iter¬ 
ative  manner.  In  that  sense,  the  approach  presented  in  Chapter  3  adds  an  important 
component  to  the  generalization  step  by  means  of  Proof  Based  Abstraction.  It  is 
interesting  to  note  that  the  approach,  as  described  in  Fig.  3.2,  is  quite  general  and 
not  restricted  to  bounding  the  number  of  iterations  of  a  loop.  For  example,  one 
can  obtain  different  kinds  of  under-approximations  by  bounding  the  range  sets  of 
the  program  variables,  or  the  stack-depth  in  a  recursive  program,  or  the  number  of 
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context-switches  in  a  concurrent  program.  It  would  be  particularly  interesting  to  ex¬ 
plore  these  ideas  for  concurrent  programs  as  such  programs  are  notoriously  difficult 
to  verify. 

Syntactic  vs.  Semantic  Abstractions.  We  have  seen  several  abstraction  and 
approximation  mechanisms  in  the  algorithms  described  in  Chapters  2  and  3.  In  the 
latter,  especially  in  our  implementation  described  in  Section  3.5,  we  use  a  combina¬ 
tion  of  a  constraint-based  method  and  program  invariants  to  obtain  an  abstraction. 
While  a  constraint-based  method  depends  on  the  input  syntactic  structure  of  the 
transition  relation,  an  invariant  is  primarily  a  semantic  artifact  -  a  formula  that 
over-approximates  the  reachable  set  of  states.  On  the  other  hand,  the  abstraction 
mechanism  described  in  Chapter  2  is  purely  semantic  -  we  use  formulas  over  the 
input-output  parameters  of  a  procedure  to  approximate  its  behavior.  It  would  be 
interesting  to  explore  a  combination  of  the  two  abstraction  mechanisms. 

Synthesizing  Ghost-code  for  Verification.  In  the  literature  on  deductive  verifi¬ 
cation,  the  term  ghost-code  is  used  to  refer  to  a  piece  of  code  that  is  added  specifically 
to  assist  in  verification  and  which  does  not  interfere  with  the  execution  of  the  orig¬ 
inal  program  (I  do  not  know  the  origin  of  the  term,  but  see  Filliatre  et  al.  [  ]  for 

a  recent  reference).  However,  to  the  best  of  our  knowledge,  there  is  no  automatic 
verification  method  that  has  a  seamless  integration  with  synthesizing  and  utilizing 
appropriate  ghost-code.  The  program  transformation  we  describe  in  Section  3.5  adds 
code  to  count  the  number  of  iterations  of  a  loop  which  is  one  of  the  simplest  kinds 
of  ghost-code.  It  would  be  interesting  to  explore  the  use  of  other,  more  expressive, 
kinds  of  ghost-code,  e.g.,  synthesizing  useful  terms  or  predicates  over  program  vari- 
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ables  (inspired  by  Predicate  Abstraction  [  ])  and  treating  them  as  first-class  objects 

as  opposed  to  artifacts  in  an  external  verification  mechanism. 

Verification  of  Evolving  Software.  Real  software  is  constantly  evolving  with 
design  modifications,  addition  of  new  functionality,  improvements  in  efficiency,  etc. 
An  exhaustive  verification  of  the  entire  software  after  every  change  can  be  very 
expensive.  It  is  interesting  to  consider  how  we  can  reuse  the  verification  efforts  from 
a  previous  version  of  the  software  for  verifying  the  current  version.  For  example, 
if  we  have  an  abstraction  of  the  previous  version  of  the  software  that  continues  to 
be  an  abstraction  of  the  current  version,  then  we  can  simply  reuse  the  invariants 
computed  in  the  past.  However,  this  is  only  the  best  case  scenario  and  there  have 
been  some  recent  attempts  at  localizing  verification  efforts  by  making  use  of  previous 
abstractions  (e.g.,  [50]).  In  general,  we  also  need  effective  ways  of  translating  the 
artifacts,  such  as  invariants,  from  previous  versions  to  maximize  reuse  and  minimize 
verification  efforts. 

Handling  Richer  Logical  Theories.  The  algorithms  we  described  in  this  dis¬ 
sertation  can  handle  programs  over  any  decidable  logical  theory.  However,  the  ap¬ 
proximation  techniques  for  existential  quantification  described  in  Chapter  2,  which 
are  essential  to  avoid  the  exponentially  growing  BMC  problems,  are  restricted  to 
Linear  Rational  Arithmetic  and  Presburger  Arithmetic.  In  particular,  if  the  theory 
has  uninterpreted  functions,  it  is  not  possible,  in  general,  to  eliminate  existential 
quantifiers  given  the  undecidability  of  first-order  logic.  This  raises  the  important 
question  of  how  we  can  efficiently  approximate  existential  quantification  in  order  to 
have  a  useful  compositional  verification  approach.  Handling  theories  with  non-linear 
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real  functions  is  another  very  important  problem  given  the  rise  of  cyber-physical 
systems.  There  have  been  recent  advancements  in  efficient  and  practical  constraint 
solving  for  such  theories  [56]  and  Bounded  Model  Checking  using  satisfiability  over 
such  theories  [57],  but  extending  the  techniques  to  scalable  unbounded  verification 
remains  an  open  challenge. 

Verifying  programs  that  manipulate  pointers  and  arrays  is  another  important 
problem  which  poses  significant  challenges  given  that  even  the  simplest  of  the  pro¬ 
grams  require  universal  quantifiers  in  the  invariants.  Given  the  rise  of  scalable  ver¬ 
ification  algorithms  for  inferring  quantifier-free  invariants  (including  the  algorithms 
developed  in  this  dissertation),  one  approach  is  to  iteratively  reduce  to  the  problem 
of  inferring  quantifier- free  invariants  [23].  This  involves  finite  heuristic  instantiation 
of  the  universal  quantifiers  with  ground  terms  and  has  been  shown  to  work  success¬ 
fully  for  small  textbook  examples  in  the  above  reference.  However,  scaling  such  an 
approach  to  handle  realistic  programs  remains  a  challenge. 

Efficient  Probabilistic  Analysis.  With  the  growing  complexity  of  software,  a 
practical  alternative  for  scaling  verification  technology  is  to  focus  on  probabilistic  cor¬ 
rectness  guarantees  as  opposed  to  ensuring  correctness  in  all  scenarios.  However,  such 
techniques  are  significantly  under-developed  when  compared  to  non-probabilistic 
analyses.  The  algorithms  presented  in  Chapters  5  and  6  are  the  first  complete  al¬ 
gorithms  for  compositional  reasoning  in  order  to  deal  with  the  state-space  explosion 
problem.  However,  real  software  deals  with  infinite  data  types  (or  practically  infinite, 
if  physical  limitations  of  the  machine  are  taken  into  account).  It  would  be  interesting 
to  explore  SAT/SMT-based  methods  for  probabilistic  reasoning.  While  there  exist 
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techniques  for  Bounded  Model  Checking  of  probabilistic  programs  (e.g.,  [26]),  we 
are  not  aware  of  unbounded  verification  algorithms.  This  would  require  new  expres¬ 
sive  logics  for  proofs,  appropriate  proof-systems,  algorithms  for  constraint  solving  in 
presence  of  probabilities  (finding  suitable  problem  formulations  as  well  as  solutions), 
etc. 
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